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Knowledge production, consumption and impact: 
policy indicators for a changing world 


Ian Rowlands 


Keywords Information management, 
Knowledge processes, Research, User studies 


This paper provides a high-level overview of some of 
the main research themes and preoccupations that 
are reported in this special ciber issue of Aslb 
Proceedings. The research activities of ciber are 
drawn together in the quest for a better 
understanding of the policy implications of large- 
scale knowledge production systems against the 
backdrop of profound technical change, uncertainty 
over business models, and new forms of consumer 
behaviour. The paper presents a series of conceptual 
frameworks that aim to contextualise ciber’s work in 
bibliometrics, cybermetrics, research evaluation, 
scholarly communication, user studies, publishing 
strategies and policy analysis. The transparency that 
metrics can bring to the evaluation debate and the 
pivotal role of human information behaviour in 
determining those metrics, are discussed. 


Information metrics and user studies 
Carol Tenopir , 
Keywords User studies, Methods, Methodology 


Three questions — what can be studied; how can 
studies be done; and what can be measured — drive 
research methods and help to identify information 
metrics for user studies. User studies can investigate 
user needs, search strategies, or preferences. 
Observing and asking, the two main methods for 
conducting user studies, yield quantitative and 
qualitative data through studying patterns of 
behavior and insights into motivation. Ciber (Centre 
for Information Behaviour and the Evaluation of 
Research, City University, London) is in a good 
position to continue supporting information user 
behavior studies that use a variety of methods to 
gather both qualitative and quantitative data and 
help establish consistent metrics. 


(Mis)understanding the digital media revolution 
Richard Withey 


Keywords Online retrieval, Information services, 
Consumer behaviour, Brand loyalty, 
Digital communications 


Describes recent confusion about the nature of the 
digital revolution and the impact of this on 
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commercial and academic behaviour. Illustrates that 
the nature of the digital revolution in consumer use 
of information has shown phenomenal uptake in 
usage, if not revenues and that this has already 
begun to change the nature of consumer behaviour. 
Argues that the power of the Web to disintermediate 
and provide scale economies could well apply to 
STM publishing models and that consequential 
changes could be significant. 


Digital information consumers, players and 
purchasers: information seeking behaviour in the 
new digital interactive environment 


David Nicholas, Tom Dobrowolski, Richard Withey, 
Chris Russell, Paul Huntington and Peter Williams 


Keywords Digital communications, 
Electronic commerce, End-user computing, 
Information services, Interaction 


Sketches the key characteristics of the newly 
information enfranchised general public (the 
digital information consumers). Portrays the digital 
consumer as all-conquering/powerful, short on 
attention, promiscuous, untrusting and — above all 
— interested in speed of delivery. Argues for a 
fundamental re-think of the concept of the 
information “user”. The Web, search engines etc. 
are creating a level-playing field and a homogeneity 
which results in academics behaving more like the 
general consumer and the general consumer 
behaving more like an academic. Considers the 
overall outcomes and benefits of information - 
acquisition. 


Beyond outputs: new measures of biomedical 
research impact 


Grant Lewison 


Keywords Information science, Mass media, 
Research, International standards 


Biomedical research evaluation has traditionally 
been based on analysis of outputs and their 
citations by other papers. However we should try to 
map the routes by which research actually 


_improves patient care and reduces illness, and 


develop indicators for them. We must allow for the 
lengthy time-scales involved and the importance of 
researchers being physically close to healthcare 
professionals, whose practice can be improved 
through international and governmental f 
regulations and through approved guidelines. Each 
of these will depend on a body of research 
evidence. We must also evaluate the effects of 
research on policy makers and the public, who 
often learn about it through the World Wide Web 
and through the mass media, particularly 
newspapers. The latter provide a major 
bibliometric resource but one that needs to be 
tapped in individual countries using common 
standards in order to provide internationally- 
comparable indicators. 
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Digital information provision via interactive 
television: understanding the digital consumer 


Barrie Gunter 


Keywords Digital communications, 
Information retrieval, Television, Interaction, 
Government departments, Dissemination, Health 


To achieve its objective of having virtually all its 
services (central and local) online by 2005, the 
Government must close the “digital divide”, i.e. 
provide access for those sections of society not yet 
linked to the electronic, computer-mediated 
communications environment. There is a need to 
consider user psychology as well as demography, as 
shown by research that has evaluated the use and 
effectiveness of health information and advice 
services provided via the television. The assumption 
is that simply providing access to online public 
services via the country’s most widespread and 
popular mass medium will guarantee their success, 
but this assumption is naive in terms of its 
understanding of viewer psychology. To ensure 
success, it is essential to know more about the 
usability and application effectiveness of interactive 
services provided through television. This paper 
explores key concepts of platform availability, use 
and access, service and information access, service 
use and application effectiveness. 


Comparing the use of two DiTV transmission 
services: same service, different outcomes 


Paul Huntington, David Nicholas and Peter Williams 


Keywords Digital communications, Television, 
Metrics, Health service, United Kingdom, 
User studies 


NHS Direct Digital was available to almost exactly the 
same number of users in both Hull and London via 
two different transmission DiTV companies: Kingston 
Interactive Television (KIT) and HomeChoice. This 
paper employs metrics based on server generated logs 
to compare use and user differences between the two 
services. Server logs that record user activity on a real- 
time and-continuous basis were made available and 
used to make comparisons between the two services. 
The data were collected for both services over a 
similar period, February 2002 to May 2002. During 
this period the NHS Direct Digital service was visited 
by approximately 3,000 people and more than 5,000 
NHS Direct videos were viewed. Making the study 
probably the largest analysis of health videos ever 
conducted. 


Home electronic health information for 
the consumer: user evaluation of a DiTV 
video-on-demand service 


Peter Williams, David Nicholas and Paul Huntington 


Keywords Digital communications, Video, 
Information services, Education, Health care, 
User studies 


Reports on a qualitative study exploring a highly 
innovative digital TV pilot service. Focus groups of 
medical and health information professionals, 
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“condition-specific” subjects and “general 
interest” volunteers were recruited to obtain views 
about the videos generally, in terms of their content 
and presentation. Issues such as terminology used 
and the appropriateness of using video, to deliver 
various types of information were also explored; 
the latter in the light of log statistics indicating 
relatively low use of videos as compared to text- 
based services. Results indicated that those who 
used the services tended to be avid seekers of 
information, who saw the service as a first port of 
call to obtain general information for wider 
consultation later. Differences emerged between 
medical professionals and the lay public regarding 
the use of language and involvement of patients in 
the videos. Text was regarded as a better medium 
for conveying some types of information, such as 
simple facts that might need to be consulted on a 
regular basis. 


Modelling the diffusion of specialised knowledge 
Julian Warner 


Keywords Dissemination, Research, Science, 
Diffusion, Knowledge 


This paper reviews developments from a study of 
the reception of the Research Assessment Exercise 
1996. Research evaluation, including the value of 
citation analysis and the responsibilities attaching 
to publication of citation analyses, is considered. 
The distinctions made in the communication 
model for analysing reception, between 
dissemination and diffusion and between esoteric 
and exoteric media and communities, are 
developed further. Information transfer is 
represented as an explicable process. Possible 
further developments, including the appropriate 
relation to disciplines with related interests in the 
social communication of knowledge, are 
anticipated. 


Digital journals, Big Deals and online searching 
behaviour: a pilot study 


Dawid Nicholas, Paul Huntington and 
Anthony Watkinson 


‘Keywords Digital communications, 
Libraries, Electronic publishing, Online retrieval, 
Journal publishing 


Evaluates, through deep log analysis, the impact of 
“Big Deal” agreements on the online searching 
behaviour of users of the Emerald digital library 
Web site, which provides access to more than 150 
journals in the fields of business and information 
science. The purpose of the evaluation was to map 
the online information seeking behaviour of the 
digital library user and to see whether those signed- 
up to a Big Deal arrangement behaved any 
differently from the others. In general they did. 
The real surprise proved to be the strong consumer 
traits of the library’s users. Research reported 
here refers to the first stage of a three-stage 
research project. 
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SSS SS 
This paper provides a high-level overview of some of the 
main research themes and preoccupations that are reported 
in this special ciber issue of Aslib Proceedings: New 
Information Perspectives. The research activities of ciber are 
drawn together in the quest for a better understanding of 
the policy implications of large-scale knowledge production 
systems against the backdrop of profound technical 
change, uncertainty over business models, and new forms 
of consumer behaviour. The paper presents a series of 
conceptual frameworks that aim to contextualise ciber's 
work in bibliometrics, cybermetrics, research evaluation, 
scholarly communication, user studies, publishing strategies 
and policy analysis. The transparency that metrics can bring 
to the evaluation debate and the pivotal role of human 
information behaviour in determining those metrics, are 
discussed. 
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Background to this special issue 


All the papers in this special double issue of 
Aslib Proceedings: New Information Perspectives 
are concerned with knowledge production, 
consumption and impact. They were first 
presented at a symposium held at The 
Bedford Swan Hotel, 24-26 July 2002, and 
represent the first corporate utterance of a 
new research group based at City University: 
ciber, the Centre for Information Behaviour 
and the Evaluation of Research. We are very 
grateful to Elsevier Science Limited and The 
Wellcome Trust for supporting the event, 
both financially and in kind. 

ciber is a policy-led, data-driven, think tank, 
which engages with issues at the interface 
between today’s value-for-money, 
measurement-driven culture and the knowledge 
economy. The Centre’s approach is highly 
interdisciplinary and draws on expertise in 
information science, bibliometrics, scholarly 
communication, research evaluation, Web 
monitoring, science policy, publishing 
strategies, new media, user studies, human- 
computer interaction, audience research and 
journalism. Current research projects include 
evaluations of consumer health information on 
digital platforms, studies of medical research 
outputs and their funding, young people’s use 
of mobile phone for information delivery, the 
reporting of biomedical research in the 
newspapers, and the development of useage 
indicators for academic journal publishers and 
aggregators. These projects span the full 
breadth of the knowledge cycle, from research 
funding to impact evaluation. All are concerned 
with information metrics and indicators in areas 
of immediate application in public and private 
policy, with a distinct bias to health and 
healthcare. The aim of this paper is to 
contextualise the papers in this special issue 
within a broader and more comprehensive ciber 
framework so that the many common themes 
and links can be more readily appreciated. 


A policy-led, data-driven research 
agenda 


Much of ciber’s research is led by the needs of 
decision makers in the public and private 
sectors who are striving to understand how to 
harness the knowledge production and 
consumption cycle to their best advantage. 
Central to ciber’s research philosophy is the 
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view that we can only understand the 
dynamics of this cycle properly if we take a 
balanced view of the interlocking 
relationships between technology, 
information behaviour and policy and how 
these forces shape knowledge production, 
consumption and impact (Figure 1). 

Figure 1 represents a virtuous circle where 
policy questions drive grounded data 
collection and analysis, reflection, theory 
building and the development of new research 
questions and new indicators with which to 
tackle them. In the area of scholarly 
communication, for example, recent ciber 
work has addressed the notion of what journal 
“impact” really means and has proposed a 
new set of citation-based measures for 
representing the diffusion of ideas through the 
scientific literature (Rowlands, 2002). In an 
era of profound changes in scholarly 
communication, we need a much broader 
range of indicators to capture the subtleties of 
journal publication and reception than the 
bibliometrics community has hitherto 
recognised. Now that the user, previously the 
ghost at the bibliometrics feast, can be 
represented by the logs left behind by their 
browsing, searching and downloading 
behaviours (see the paper by Dave Nicholas 
and colleagues on “Big Deals” in academic 
publishing in this issue, pp. 84-109 (Nicholas 
et al, (2003b)), we are seeing the beginnings 
of a much more holistic understanding of the 
dynamics of the whole scholarly cycle. 


Figure 1 A dynamic research agenda 
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In some ways, the kinds of questions that 
ciber researchers ask will be very familiar to 
certain readers of this journal, especially those 
with an active interest in the library 
performance indicators movement. What 
resource inputs are available to meet the 
information needs of a given user population? 
How efficiently are those resources deployed? 
What is the ultimate value, what is distinctive, 
what is meaningful, what we are trying to do 
with those resources? How will users interact 
with our offerings, shaping, forming and 
transforming our ways of understanding their 
needs? How cost-effective are our investments 
in these activities? 

What is different and unique about ciber is 
that it frames these questions not at the level of 
measuring the efficiency or effectiveness of the 
interlending services at the University of 
Poppleton, but at the level of corporate or 
national policy and within a context of a 
society where information is commonly 
recognised as a key lever of change. We take 
the view that there are important questions of 
public interest that need to be asked of 
scholarly communication systems, of public 
healthcare systems, of initiatives in 
e-government (see, for example, Barrie 
Gunter’s discussion of the attempts of the Blair 
Government to close the “digital divide” in 
this issue, pp. 43-51 (Gunter, 2003)). 
Information products and services can now be 
configured simultaneously on the Web, on 
optical media, on handheld devices and mobile 
‘phones, on digital TV and even as print on 
paper. Information behaviour is changing as 
new opportunities present themselves, raising 
enormously significant issues. In this issue, we 
begin to grapple with some of these large scale, 
Big Picture, policy issues that effect us all. 
Measurement, and the implied transparency 
that metrics offer, is the common 
denominator. 

ciber members have a common 
interest in mining large data sets with a view 
to extracting policy-relevant findings. 

The policy data pyramid (Figure 2) is a 
model that locates the production of 
indicators and other summary data within an 
action context that is designed to inform 
other specialists, policy makers and the 
wider public. Much of what ciber does can 
be understood in terms of reducing the 
complexity in high volume, sometimes 
ambiguous data, like Web logs, so that 
underlying truths can at least be glimpsed. 
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Figure 2 The policy data pyramid 
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The advantages of objective and unobtrusive 
quantitative techniques such as bibliometrics 
are obvious, but they can potentially lead the 
researcher into a cul-de-sac where analysis is 
driven by what can practically be done with 
the data at hand, rather than responding to 
the kinds of questions that ought ideally to be 
asked. In essence, data mining is often tools- 
led rather than policy-driven, and this 
suggests some rather awkward questions: 
* What do policymakers do with research 
information and indicators? 
* How critical are they of their limitations? 
* What kinds of indicators and research do 
policy makers really want? 
+ What would they ask for if they could 
start with a clean sheet? 


In other words, what is measured and what 
ought to be measured? 

Equally important, what are the 
information needs, information seeking 
behaviour and preferences of policy makers? 
How do they use policy-relevant information 
and research? These are important questions 
and so the attitudes and information 
behaviour of policy makers become legitimate 
topics for ciber research in their own right 
(see Strachan and Rowlands, 1997). 

In this issue, Julian Warner (pp. 75-83) 
points out some of the limitations of indicator 
systems in a policy context, not least the 
danger of confusing description and value. 
Indicators are artificial constructs. As he 
points out, “citation counts [for example] are 
no more, and no less, objective than any other 
collective human activity” (Warner, 2003). 
This is a useful insight in the field of biblio- 
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and cybermetrics, which have generally rather 
neglected the decision focus of the indicators 
they create. It is imperative that information 
metrics should be developed with a view to 
their relevance in decision making. They 
should aim to answer questions as directly as 
possible and form the basis for policy 
recommendations. Metrics and other data 
that do not perform this function become an 
end in themselves, offer no new insights, and 
quickly become isolated data geared to 
impress rather than inform. It is for these 
reasons that ciber places a great deal of 
emphasis on trying to understand the 
information behaviour that underpins the 
knowledge production cycle and the policy 
conditions within which it operates. 


Beyond outputs to outcomes and 
impacts 


Many of the frameworks traditionally used in 
evaluating library performance or national 
science policy emphasise resource inputs into 
the system (e.g. numbers of volumes held, 
national expenditures on civil R&D) and 
processes (e.g. costs of book processing, PhD 
completion rates) which address the efficiency 
with which those resources are utilised. The 
economic thinking, perspectives and 
terminology behind these indicator systems 
are by no means wholly appropriate or 
adequate when thinking about knowledge 
production, consumption and impact. 

The economics of information are poorly 
understood, not least in relation to its public 
good and transformative characteristics. It all 
depends on whether you think information is 
a verb or a noun, whether information is just 
something you produce and consume or 
something that shapes how and what you 
think. There may be better models and 
frameworks, perhaps based on the 
communication and diffusion of specialised 
knowledge (see the paper by Julian Warner in 
this issue (pp. 75-83) which models 
information transfer through various esoteric 
and exoteric media (Warner, 2003)), or which 
reconceptualise our understanding of “user” 
not as a passive destination but as an active 
player in the consumption and transformation 
of knowledge (see Dave Nicholas and 
colleagues in this issue, pp. 23-31, on the 
notion of digital information players 
(Nicholas et al., 2003a)). 
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Metrics can only provide an understanding 
of the knowledge production cycle on the 
physical plane, such as by counting numbers 
of documents, citations or downloads. The 
main purpose of knowledge production, 
creating understanding, can only be glimpsed 
indirectly. For this reason, metrics research 
needs to be triangulated with observation, 
survey and ethnographic techniques to 
provide the full picture (Pullinger and 
Baldwin, 2002). These issues and the need for 
a more holistic approach rooted in human 
cognition are explored in more depth in the 
paper by Carol Tenopir (2003) in this issue 
(pp. 13-17). Unexpected findings are 
reported in several papers in this issue when 
the real-life preferences and behaviours of 
information consumers are studied in depth 
(see the papers by Huntington et al. (2003), 
pp. 52-63, and Williams ez al., 64-74) 
evaluating the public’s use and perceptions of 
consumer health information delivered on 
digital TV platforms. 

Much more fundamental work needs to be 
done to develop a model of the knowledge 
production cycle that transcends “old 
economy” thinking, one that can more easily 
relate inputs and outputs to consumption, 
impact and changed human behaviour. In the 
meanwhile, ciber researchers have still found it 
useful to use a simple model to organise our 
thinking about the different kinds of indicators 
and other data that are needed to develop 
better, evidence-led, policies (Figure 3). 

In this model, knowledge production is 
conceptualised as a “black box” that takes in 
inputs (e.g. research funding, skills, new 
ideas) and transforms them into outputs. An 
output is a tangible result of an activity: so, an 
academic research project may produce 
outputs in the form of a series of papers and 
patents. Since we are dealing primarily with 
informational outputs, the use and 
consumption of knowledge becomes a 
particularly important focus for ciber. 


Figure 3 Knowledge production cycle indicators 
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Information consumption thus features as a 
strong theme in this issue. 

An outcome is the principal goal of an 
activity, in this hypothetical case, a new 
invention: an outcome may thus incorporate a 
number of outputs. Impacts are quite different 
from outputs and outcomes. Outputs and 
outcomes are the discernible direct results of 
knowledge creation, as in the examples given: an 
article, a patent or an invention. Impacts are the 
broader and less immediate consequences that 
follow on from a paper or an invention, thus 
television has an impact on learning; mobile 
phones have an impact on teenage lifestyles. 
Impacts may occur over a long period of time. 
Work on information impact indicators is an 
exciting but underdeveloped area, but one in 
which ciber is increasingly being involved, and 
the reader of this issue is particularly directed to 
Lewison’s programmatic paper ‘pp. 32-42) on 
the “downstream” outcomes and impacts of 
biomedical research CLewison, 2003). He asks 
the deceptively simple question, “what are the 
links between research and better health?”. 

The integrated nature of the knowledge 
indicators model is useful in that it presents 
the big picture. For example, in the higher 
education sector, it becomes possible to think 
seamlessly about the policy inputs that 
facilitate knowledge production (like funding, 
access to library resources), outputs (like 
peer-reviewed research publications), 
consumption of supporting information 
services (e.g. JISC) and downstream research 
impact and quality measures, such as the 
national Research Assessment Exercises. We 
believe that this model is scaleable from a 
single information application right up to a 
national science system. 


The research programme 


Two overlapping, research themes have 
developed around ciber’s early activities, 
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partly by design, partly adventitiously around 
some of the larger-scale projects: evaluating 
knowledge production systems, and 
understanding information behaviour. 


Evaluating knowledge production 
systems 

This research theme is concerned with the 
chains that link authors with readers through 
scholarly communication systems, that link 
health promotion agencies with digital health 
consumers through kiosks and other new 
technology platforms and, to express this idea 
at a more general level, the chains that link 
information creation, packaging, distribution 
and consumption. Essentially, this theme is 
concerned with understanding the digital 
transition and its impacts on knowledge 
dissemination. Completely new forms of 
knowledge production are emerging around 
electronic communication technologies such 
as the Internet. The digital transition is 
characterised by new kinds of documents (e.g. 
mobile text messages for health promotion), 
new kinds of repositories (e.g. e-print archives, 
digital libraries), new kinds of behaviour (e.g. 
hypertext linking, information foraging) and 
new kinds of business models (e.g. Big Deals, 
pay-per-view). These developments are 
radically reshaping the whole information 
landscape and creating fresh opportunities for 
revisiting “old” bibliometric models in the new 
networked information space. 

The evaluation of these systems requires a 
fresh approach, one based on massive data 
collection, observation and grounded analysis 
since the models of the past are no longer a 
sure guide to the future. As we saw earlier, the 
needs of policy makers are pulling ciber well 
beyond the evaluation of knowledge 
production systems per se to trying to 
understand the wider outcomes and impacts of 
those systems. So, for example, do investments 
in health promotion impact on health 
outcomes, and can information provision 
change human behaviour? These questions 
suggest the need for a broader and more fluid 
range of methodological tools than researchers 
in the bibliometrics or cybermetrics 
communities have previously considered. 

A key stumbling block in the evaluation of 
knowledge production is what is meant by 
“quality” and how the notion of quality can be 
glimpsed and condensed down to provide 
useful indicators. Ultimately, we have to 
concede that quality is unmeasurable. 

Who is the greater opera composer, 
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Britten or Verdi? Who the greater theologian, 
Martin Luther or Thomas Aquinas? Who has 
contributed more to information retrieval 
theory, Karen Spark-Jones or Steven 
Robertson? While these may be interesting 
topics to debate, any notion of absolute quality 
is unworkable and better left to the 
philosophers and historians of the future. 

We might do better to rethink quality as 
fitness-for-purpose, as a measure of 
satisfaction. This is how quality is commonly 
understood in contemporary management, 
This insight might lead us to the following 
seven-point model, or something like it, for 
understanding the quality of research (Table D. 
The model is based on the idea of a knowledge 
production life cycle, from an initial idea to its 
ultimate fulfilment, and in its current 
incarnation, the model is geared towards a 
better understanding of peer-reviewed 
scholarly journal publishing. However, we 
believe that the essential points of this model 
may be generalisable to other forms of 
knowledge production, such as new media 
publishing and work is in progress to develop a 
series of “sectoral” models. 

As they progress through this cycle, 
information and knowledge demonstrate: 

* increasing value; 

* increasing consensus and social 
acceptance; 

* increasing persistence; and 

* increasing socio-economic impact. 


The relevance of this model to ciber’s 
activities can be exemplified by reference to 
one of our flagship projects, the Research 
Outputs Database (ROD). ROD is funded by 
The Wellcome Trust and a consortium of 
more than 30 medical charities and research 
councils. It is a database tool, based on ISI 
data, containing bibliographic details of all 
significant peer-reviewed biomedical research 
papers published by UK or Irish authors since 
1988. Each paper in the database contains 
data elements, which relate to funding 
acknowledgements (an input indicator), 
publication details (outputs) and citation 
measures (an indicator of consumption. by 
other researchers). ROD methodologies are 
being extended all the time, and Grant 
Lewison (2003) has been working especially 
on developing measures of outcome (for 
example, the citation of research papers in 
clinical guidelines, and the explicit 
incorporation of research findings into 
bedside treatments). 
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Table I The knowledge product life cycle 


Research is ... 


Whose needs are satisfied? What are we dealing with? 





Supported 


Published 


Cited 


Read 


Influenced 


Adopted 


Established 
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Notes 





Grant givers Individual ideas (leading to 
research proposals leading to 


research grants) 


Journal editors/referees Publications 


Research active scholars Cited publications 


Research active scholars/ 
other researchers and 
teachers/students 


Readership 


All in the above boxes, 
plus policy makers 


Socially accepted ideas 


All in the above boxes, Socially accepted technologies 





In an abundance of individual ideas and scarce resources to 
support them, the best ideas rise to the top through a 
combination of self-censorship and by gaining the support of 
research funders 


The best papers are published if they satisfy the requirements 
of pears. Almost by definition, this implies acceptance that the 
ideas are worthy of wider circulation 


The most useful papers are accepted by other research active 
scholars through citation: a direct measure of the degree to 
which the new ideas satisfy the needs of the research field 


The papers with the broadest reach are read well outside the 
immediate field of study: by others within and beyond the 
discipline 

Some of the ideas in these papers are so powerful that they 
have conceptual influence (they change the way others think), 
or instrumental influence (they change what others do). By 
virtue of the exposure they have received in getting this far, 
they have moved from being an individual's ideas to socially 
accepted ideas 


Sometimes, these ideas find a practical social use and become 


plus practitioners, 
entrepreneurs, consumers 


Whole cultures 


Understanding information behaviour 
This research theme is about the world of the 
empowered information consumer, 
interactivity and the next generation of 
knowledge platforms. The current transition 
from print to an increasingly bewildering 
array of electronic sources, distribution 
technologies, and new information 
applications suggests some crucial questions 
for scholars, policy makers, investors and 
citizens alike — from how to price information 
to how to evaluate quality in the permissive 
publishing environment of the Internet. The 
statistics of the new digital interactive 
environment are pretty amazing (this issue, 
passim). Information and communication 
technologies are reconfiguring every aspect of 
the knowledge cycle: 


Electronic scholarly communication is reaching 
critical mass, and we are witnessing qualitative 
and quantitative changes in the ways that scholars 
communicate with each other for informal 
conversations, for collaborating locally and over 
distances, for publishing and disseminating their 
work, and for constructing links between their 
work and that of others ... The cycle of scholarly 
activities is blending into a continuous, looping 
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Socially accepted orthodoxies 


embedded in practices, technologies or markets 


as part of the fabric ... until challenged 


flow, as people discuss, write, share, and seek 
information through networked systems 
(Borgman and Furner, 2002). 


Excited by what has now become possible, 
participants in the creation of the new digital 
library often overlook fundamental (often 
implicit) aspects of the paper paradigm that still 
need to be maintained in the digital universe. In 
the realm of scholarly and scientific 
communication, thus issues of first publication, 
version control, referencing version control, 
referencing, stable and immutable archives, as 
well as strategic issues blurring the distinctions 
between the information players all have to be 
reconsidered (Mabe, 2001). 


Despite the “hard” scientific methodologies 
and terminologies employed by 
infometricians, it should never be forgotten 
that knowledge production, consumption and 
impact are social phenomena that are 
fundamentally shaped and determined by 
human behaviour: affective, cognitive and 
sensorimotor aspects all play a role here. 
Humans are creatures of habit, leaving behind 
them trails of information behaviour evidence 
that often reveal striking statistical 
regularities. These patterns and the 


Eventually, papers are no longer published, read, cited or 
influential (like gravitational theory) they are simply accepted 


rj 
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underlying human behaviours that give rise to 
them, form the subject matter of information 
metrics. Bibliometric “laws” are human- 
centred distributions, not natural laws. They 
reveal the behaviour patterns of populations 
of information users and producers, their 
likes, dislikes, reading habits and perceptions 
(Mabe and Amin, 2002) and they offer a 
unique glimpse into the social psychology of 
knowledge production and consumption. 

Recent technical developments in the 
knowledge production cycle suggest some 
fundamental overarching questions. How 
much has human behaviour really changed? 
Are we witnessing a revolution, or a 
co-evolution of technology and behaviour? 
(See the paper by Withey (2003) in this issue, 
pp. 18-22.) Although we do not yet have 
many clearly formulated questions, let alone 
answers, it is blindingly evident that we 
cannot understand information processes 
without first understanding human 
behaviour. We need a much better 
understanding of the reasons how and why 
people browse, forage, interact and consume 
and subsequently utilise electronic 
information. We also need a better 
understanding of the reward systems that 
motivate and shape information and its 
production. The importance of human 
behaviour in the context of ciber’s activity 
manifests itself at many levels. We know for 
example that league tables, performance 
indicators and other forms of evaluation have 
a profound impact on the way that 
organisations and employees behave. Talk to 
a teacher about Ofsted inspections or to a 
medical school librarian about journal impact 
factors! More fundamentally, bibliometricians 
have suspected for many years that their 
straight log-linear plots, hyperbolic 
distributions and “laws” must be grounded in 
the habits and regularities of the interactions 
between authors, referees, journal editors and 
readers. Similar patterns are emerging from 
studies of Bradfordian subject concentration 
on electronic bulletin boards, or from the 
Zipfian surfing behaviour of people navigating 
the World Wide Web (Rowlands, 2000). 


In summary 


Bibliometrics research, after a long period in 
the doldrums, is becoming more and more 
relevant to questions of public and corporate 
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policy in an information- and outcomes- 
obsessed society. The focus is shifting from an 
introverted and self-reflexive concern with 
simply summarising and condensing the 
content aspects of large databases (text 
mining) to an approach that explores 
profound questions about information 
behaviour, motivation and value. Central to 
ciber’s research ethic is that bibliometrics, 
and latterly cybermetrics, are keys to 
unlocking the secrets of information 
behaviour. The context of information 
metrics research has broadened to encompass 
knowledge production, consumption and 
impact as primarily social rather than 
technical phenomena. It is wising up. The 
opportunities for differentiating information 
delivery by media, by demographic 
characteristics, by retrieval and browsing 
preferences, and by business model suggested 
by the research findings presented in this issue 
raise some fascinating research questions for 
the future with a policy resonance that has not 
previously been tackled within the 
information science tradition. 


Guest Editor's note 


In addition to the authors of the papers 

appearing in this issue, the following took part 

in the symposium: 

* Nadia Azzouzi (The Wellcome Trust); 

* David Brown (The British Library), 
Chair of session; 

* Josie Maria Griffiths (University of 
Pittsburgh); 

e Don King (University of Pittsburgh); 

* Eti Herman (University of Haifa), 
Rapporteur; 

e Michael Mabe (Elsevier Science Ltd); 

e Michel Menou (City University), 
Rapporteur; 

* David Pullinger (UK Census Office); 

e Lyn Robinson (City University); and 

e Tain Stevensen (City University). 


References 


Borgman, C. and Furner, J. (2002), “Scholarly 
communication and bibliometrics”, Annual Review 
of Information Science & Technology, Vol. 36, 
pp. 1-40. 

Gunter, B. (2003), “Digital information provision via 
interactive television: understanding the digital 


Knowledge production, consumption and impact 


Aslib Proceedings: New Information Perspectives 





Jan Rowlands 


consumer", Aslib Proceedings, Vol. 55 Nos 1/2, 
pp. 43-51. 

Huntington, P., Nicholas, D. and Williams, P. (2003), 
“Comparing the use of two DiTV transmission 
services: same health information service, different 
outcomes”, Aslib Proceedings, Vol. 55 Nos 1/2, 
pp. 52-63. 

Lewison, G. (2003), “Beyond outputs: new measures 
of biomedical research impact”, Aslib Proceedings, 
Vol. 55 Nos 1/2, pp. 32-42. 

Mabe, M. (2001), “Digital dilemmas: electronic challenges 
for the scientific journal publisher”, Aslib 
Proceedings, Vol. 53 No. 3, pp. 85-92. 

Mabe, M. and Amin, M. (2002), “Dr Jekyll and Dr Hyde: 
author-reader asymmetries in scholarly publishing”, 
Aslib Proceedings, Vol. 54 No. 3, pp. 149-57. 

Nicholas, D., Dobrowolski, T., Withey, R., Russell, C., 
Huntington, P. and Williams, P. (2003a), “Digital 
information consumers, players and purchasers: 
information seeking behaviour in the new digital 
interactive environment”, Aslib Proceedings, Vol. 55 
Nos 1/2, pp. 23-31. 

Nicholas, D., Huntington, P. and Watkinson, A., (2003b) 
“Digital journals, Big Deals and online searching 
behaviour: a pilot study”, Aslib Proceedings, Vol. 55 
Nos 1/2, pp. 84-109. 


12 


Volume 55 - Number 1/2 - 2003 - 5-12 


Pullinger, D. and Baldwin, C. (2002), Electronic journals 
and User Behaviour, Deedot Press, Cambridge. 

Rowlands, |. (2000), “Who can count the dust of Jacob? 
From bibliometrics to cybermetrics”, in 
Nicholas, D. and Rowlands, |. (Eds), The Internet: 
Its Impact and Evaluation, Aslib, London, 
pp. 114-30. 

Rowlands, |. (2002), “Journal diffusion factors: a new 
approach to measuring research influence”, Aslib 
Proceedings, Vol. 54 No. 2, pp. 77-84. 

Strachan, J. and Rowlands, |. (1997), “Information for 
policy-making”, in Rowlands, |. (Ed.), Understanding 
Information Policy, Bowker-Saur, London, 
pp. 61-73. 

Tenopir, C. (2003), “Information metrics and user studies”, 
Aslib Proceedings, Vol. 55 Nos 1/2, pp. 13-17. 

Warner, J. (2003), “Modelling the diffusion of specialised 
knowledge”, Aslib Proceedings, Vol. 55 Nos 1/2, 
pp. 75-83. . 

Williams, P., Nicholas, D. and Huntington, P. (2003), “Home 
electronic health information for the consumer: user 
evaluation of a DiTV video-on-demand service”, Aslib 
Proceedings, Vol. 55 Nos 1/2, pp. 64-74. 

Withey, R. (2003), “(Mis)understanding the digital media 
revolution”, Aslib Proceedings, Vol. 55 Nos 1/2 , 
pp. 18-22. 


ro 


Emerald 





Information metrics 
and user studies 





Carol Tenopir 


[The author = 


Carol Tenopir is a Professor at the School of Information 
Sciences, University of Tennessee, Knoxville, Tennessee, 
USA. E-mail: ctenopir@utk.edu 


a 


User studies, Methods, Methodology 


Abstract 


Three questions — what can be studied; how can studies 
be done; and what can be measured — drive research 
methods and help to identify information metrics for user 
Studies. User studies can investigate user needs, search 
strategies, or preferences. Observing and asking, the two 
main methods for conducting user studies, yield 
quantitative and qualitative data through studying 
patterns of behavior and insights into motivation. ciber 
(Centre for Information Behaviour and the Evaluation of 
Research, City University, London) is in a good position to 
continue supporting information user behavior studies 
that use a variety of methods to gather both qualitative 
and quantitative data and help establish consistent 
metrics. 


| Electronic access | access 


The Emerald Research Register for this journal is 
available at 
http://www.emeraldinsight.com/researchregister 





The current issue and full text archive of this journal is 
available at 
http:/Awww.emeraldinsight.com/0001-253X.htm 





Aslib Proceedings: New Information Perspectives 
Volume 55 - Number 1/2 - 2003 - pp. 13-17 

© MCB UP Limited - ISSN 0001-253X 

DOI 10.1108/00012530310462661 


Introduction 


Information metrics in user studies can be 
examined from the specific viewpoint of 
identifying and listing individual measurable 
data points (variables) or from the broader 
viewpoint of understanding how the research 
method selected determines what can be 
measured. This paper takes the latter, broader 
approach. Wang (2001) takes an even 
broader approach that focuses on research 
methods and methodologies. There are three 
fundamental questions related to this broad 
approach to user studies and information 
metrics: 
(1) What can be studied? 
(2) How can each of the items identified be 
studied? 
(3) What can be measured? 


Although these may seem rather simplistic at 
first glance, the answers to these questions 
drive research and research methodologies. 


Background 


To understand what can be studied requires 
some brief background on user studies in 
information retrieval. Research on human 
factors in information access, retrieval and use 
has gained popularity over the last two and a 
half decades, in particular in the last decade. 
This is in contrast to the earliest days of 
information retrieval, when researchers 
focused on aspects of retrieval systems other 
than the human in their attempt to evaluate 
retrieval performance or improve online 
systems. Studies focused on the vocabulary 
used in indexing and abstracting, on the type 
of text being searched (full text or 
bibliographic), and on the powers of the search 
engine. Each of these things is important to 
test, yet another important factor was being 
left out: the user. It took some time for 
researchers to realize that the person sitting at 
the keyboard might have something to do with 
the success or failure of a search and how users 
judge and use the information is as important 
as how they search for it. 

Studies of human factors in information 
retrieval came into their own in the early 1980s 
when researchers such as Fenichel, Bellardo, 
Bates, Saracevic and others began to look at 
how individual characteristics of intermediary 
online searchers might influence their searching 
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style or their online success on traditional online 
systems such as Dialog (Hewins, 1990; Harter 
and Hert, 1998). Only a few studies looked at 
end-user performance, such as the study by 
Borgman (1989) on how college students 
interacted with online public access catalogs. 

Early studies were almost always conducted 
on bibliographic databases on Boolean logic 
systems with command interfaces. They 
focused mostly on information seeking 
patterns rather than what the user did with 
the information they located. Collectively 
these studies tested many characteristics of 
individuals that might account for how people 
interact with information systems, and 
although they reached no firm consensus on 
which characteristics were most important, 
they laid the groundwork for future studies. 

A newer generation of human studies has 
focused more on end users and how they 
interact with information from the time of 
identifying an information need, through 
seeking, retrieval, and use of the information. 
These later studies cover a variety of systems, 
including the same Boolean logic commercial 
systems used in earlier studies, but also various 
Web search engines and other search systems. 

These later studies can be described as being 
of two types: first, the micro/holistic approach, 
and second, the macro/longitudinal approach. 
Holistic studies are typified by researchers 
such as Kuhlthau, Ingwersen, Wang, and 
others who are working to develop models of 
the total process of human interaction with 
information systems (Wang, 2001). 

Models, such as those by Ingwersen 
(Figure 1) and Wang (Figure 2) show human 
interaction with information systems as a 
complex multidimensional process that 
involves many aspects of an individual’s 
cognitive processes as well as aspects of the 
information system. A model is developed 
through observation and experimental studies 
and is then tested and refined. Creating better 
models of the entire information retrieval and 
use process is the focus of these holistic studies. 

Another current group of researchers 
approach human factor and information 
retrieval interaction studies from both a more 
quantitative and longitudinal approach. 
Researchers in this group of studies include 
Tenopir and King, Pullinger, Mabe, 
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studies often look at changing patterns over 
time to help predict the future and may 
predict behavior based on large numbers of 
responses to surveys or interviews, rather than 
experimental studies of relatively few 
individuals (King and Tenopir, 2001; 
Borgman, 1996; Mabe and Amin, 2002). 

Macro studies that are longitudinal most 
often survey large random samples of different 
individuals within the same user group 
(scientists within universities, for example) 
over time to make general observations or 
predictions of change (Tenopir and King, 
2000). Less commonly, smaller studies may 
follow the same individuals over time 
(Borgman et al., 1989). Macro/longitudinal 
studies allow researchers to depict broad - 
patterns of behavior and how those patterns 
may change over time, such as the amount of 
time spent reading articles and number of 
article readings by scientists, social scientists 
and engineers as depicted in Figure 3. 


What can be studied? 


The possibilities are infinite when it comes to 
studying human beings. Studies of user 
interaction with information can focus on any 
aspect of the information process and on the 
many roles a user performs. Comparing 
survey data with transaction log analysis can 
provide a double-check for measuring what 
people actually do and begin to answer the 
questions of both what people do and why. 

In the role of information seekers and users 
or readers we can study: 

e User needs/search topics (what do people 
intend to search for, how do they express 
those needs, how do those needs change 
over time?). 

* Search strategies (how do they pose 
strategies and interact with systems and 
with intermediaries, how do those 
strategies/interactions change, what 
difficulties do they encounter, how can 
publishers and librarians assist in this 
process, and searching style?). 

* Factors that affect strategies/interaction 
(how do individual differences, 
situational, emotion, tasks, etc. change, 


how and what people search for and use, 
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Figure 1 Ingwersen’s cognitive model of information seeking and IR interaction 
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Figure 2 A multidimensional model of user-Web interaction in IR 
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Figure 3 Time spent and number of article readings 
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Preferences (what do they like, what do 
they say they like?). 


In the role of author/creator of information we 
can add: 

Preferences/needs (what do they say 
motivates them to choose a source for 
publishing or disseminating their work?). 
Where do they choose to publish or 
disseminate their work (what do they 
actually do?). 

How do these factors vary by discipline, 
situation, and other factors? 


How can studies be done? 


Deciding what can be studied is just the first part 

of the problem. What can be measured is, of 

course, dependent upon the methodology used. 

There are many possible methodologies for user 

studies (see Wang, 2001 for an extensive 

overview), but there are two main methods that 

dictate what is measured. Simplistically, these 

two methods are: observing and asking. 
Observing can include such specific 

methodologies as: 

Query logs. 

Verbalization (concurrent (“think 

aloud”) or retrospective). 

Journals. 

Usability testing. 


Asking may be done through interviews or 
questionnaires and includes such methods as: 
Surveys (to a random sample or self- 
selected participants). 

Focus groups. 

Critical incident technique (asking a 
respondent to focus on a specific 
incident, such as the last time he or she 
read a research article). 

Recollection. 

Preferences/opinions. 

Thought experiment (having someone 
“bluesky” as to what if). 


What can be measured? 


Specific measures or metrics for all of these 
can be put in the simplest terms as: 

What people do. 

What people prefer. 

What people say they do. 

What people say they prefer. 

What they may do or prefer in the future. 
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What people do can be measured in several 
ways, including surveys that ask them what 
they do, experimental observations of what 
they do in specific situations or focusing on 
specific variables, case studies that observe 
selected user behavior, or transaction log 
analyses. Survey questions can ask directly 
what someone does (for example, how many 
articles, reports, and books did you read last 
year?), or use the critical incident technique 
to help them focus on a specific incident and 
answer questions in that context (for 
example, think about the last thing you read 
for work and describe it). Both approaches 
are valid for different levels of questions and 
are often mixed in the same study for cross- 
comparison purposes. 

What people say they want is less useful 
than what they actually do or will do, but has 
some value nevertheless. I have used almost 
all of these methods and am the most 
skeptical of the value of the information I get 
from studying stated preferences/opinions. I 
would much rather measure what people do 
in a given situation, rather than ask them what 
they prefer. Preferences are useful within a 
given context (i.e. do you prefer this interface 
or that one and why), but are Jess useful with 
novice respondents out of context (i.e. what 
do you want in an interface). It is, however, 
useful to measure what users actually do 
versus what they say they prefer. Focus 
groups, case studies, interviews, and surveys 
that ask participants to rank or rate specific 
items are most useful methods for measuring 
preferences. 

Well-designed human studies include 
both quantitative and qualitative data 
gathering and analysis. Researchers need to 
have both of these to provide the most 
reliable insights into information behavior. 
Although preference data can be quantified 
(asking people to rate their preferences on a 
scale of one to five, for example, and 
ranking preferences by averages), 
quantitative data from surveys or 
experiments is particularly useful for 
measuring what people actually do. 

Qualitative analysis is more useful for 
measuring what people want or say they want 
and for answering why they behave in a 
certain way. Careful reading of open-ended 
questions or probing interviews can reveal 
motivations and the differences between what 
people say they want and what they will really 
find useful (Watson, 1998). 
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Conclusions 


Whether using standard experimental 
techniques, surveys, or other observations, 
research funders and centers such as ciber 
should encourage projects that will provide 
both insights into patterns of behavior of user 
groups and those that provide insights into 
motivations. Insights into patterns of behavior 
of users and user groups is best done by 
quantitative data gathered in a rigorous way, 
while insights into motivations are best done by 
qualitative data and drawing meaning from 
those data. Recent studies have demonstrated 
that there are identifiable patterns in 
information behavior, but those patterns vary 
considerably by factors such as subject 
discipline, workplace, motivation, purpose, and 
many other factors (Tenopir and King, 2000; 
Pullinger and Baldwin, 2002). We are finding 
that all aspects of human behavior — affective, 
cognitive, and sensorimotor — have an influence 
on how people interact with information. 
Finally, research has shown that users may 
change their habits and expectations with 
changing situational factors (Wang, 2001). 

From studies conducted to date we are 
beginning to build a body of conclusions that 
will help direct information system design for 
the future and the beginnings of a set of 
consistent, user-centered metrics, but there is 
much work left to be done in the future. 
Research centers such as ciber are playing an 
important role in encouraging and conducting 
studies that use a variety of techniques, collect 
both quantitative and qualitative data, build 
on other studies, and provide a longitudinal 
view of user behavior. 
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Introduction 


The upturns and downturns of the digital 
revolution of the 1990s and early part of 
this centurv are well rehearsed. Economic 
historians will tell us in time just how 
significant the dotcom bubble was in 
influencing the nature of the subsequent 
world recession: it certainly seems to have 
given it some shape. With four false starts 
so far in the recovery cycle (as of early 
October 2002) compared to a total of five 
in the 1930s recession, this is looking less 
like a “V” shaped recession, or a “U” 
shaped one, and more like a “WWW” 
shaped one. 

Because the hype was so hyper and the 
subsequent low has been so low (particularly 
in the media industry, which could be why 
you have heard so much about it) some 
industry watchers have become a little 
confused as to what is actually going on in 
terms of digital conversion. By and large, the 
early pundits of the digital “new economy”, 
including some major consultancies, got the 
financial and economic indicators of growth 
spectacularly wrong. Interestingly, however, 
they got the numbers of users actually 
adopting these new technologies in their 
everyday business and personal lives pretty 
much right. So we now find ourselves, 
towards the end of 2002, with a digital 
economy that reaches half a billion people 
worldwide, and where in the UK over 21 
million users are online (see Table I), and 
with an industry that, in global terms, is not 
yet worth a damn. This has caused many 
consumer information industry watchers 
and providers to write off the digital 
channels as an overblown threat that has 
gone away, perhaps forever, but the key 
word in the preceding sentence is “yet”. 
Tables II-IV give some indication of the size 
of expansion of the audience for online 
information. 

Table II shows that one of the reasons that 
growth in digital use may still be being 
misrepresented is that users are, by and large, 
a bit younger than those who write about 
them. Almost 75 per cent of Web users in the 
UK, for example, are under 50. 

Table III shows that ABC1 socio- 
economic groups also still dominate“ 
adoption. Although this picture changes 
significantly for digital TV penetration in the 
UK (see Table IV). 
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Table | Online penetration in the UK 





Per cent 
Gender 
Male 59.55 
Female 40.45 
Access from: 
Home 66 
Work computer 29 
Mobile phone 4 
PDA 1 
Total number 21,392,051 


Source: Nielsen/NetRatings (2002) 


Table II Age profile of Web users 





Age Per cent of users 
2-11 4.20 
12-17 9.06 
18-20 3.59 
21-24 4.58 
25-34 20.33 
35-49 30.72 
50-54 11.73 
55-64 10.47 
65-99 5.31 


Source: \ielsen/NetRatings (2002) 


Table III Usage of the Web by socio-economic group 


Group Per cent 
AB 36 
C1 33 
C2 18 
DE 14 


Source: -Mori Technology Tracker (2002) 


Table IV Usage of the TV by socio-economic group 


Group Per cent 
AB 44 
C1 43 
C2 46 
DE 34 


Source: e-Mori Technology Tracker (2002) 


Hyper growth driven by the USA has now 
peaked and in fact Europe and Asia are now 
outstripping the growth in Internet usage in 
the USA: 


The number of people that have access to the 
Internet from their homes reached 498 million 
by the end of 2001. Fastest growth was in Asia, 
which saw a 5.6 per cent increase in home 
connections in the last quarter of 2001 
compared to the previous quarter. Next was 
Europe, which saw a 4.9 per cent quarterly 
increase, then the US, up 3.5 per cent, and 
Latir. America, up 3.3 per cent (Nielsen/ 
NetRatings, 2002). 


19 


Volume 55 - Number 1/2 - 2003 - 18-22 


Some early lessons 


Even from this brief overview, therefore, a 
picture emerges of a media delivery channel 
that reaches a significant proportion of citizens 
in the world’s developed economies; and that 
is now beginning to grow faster in Second and 
third world economies where growth is fuelled 
by affluent younger people but also where 
digital TV, in those few areas of the world in 
which it has made any impact, could become a 
key vehicle for digital take-up across all age 
and socio-economic groups. Moreover the 
evidence from the USA is that higher levels of 
broadband access from the home increase time 
spent on the Web significantly: 
“Online activity picks up in July as overall audience 
levels dip”, 20 August 2002, Patrick I. Thomas, 
author. Overall traffic for the US at-home and at- 
work active Internet universe fell slightly in July, 
while page view and time spent metrics crept 
higher. Falling mortgage rates and a healthy home/ 
apartment-buying season culminated in a 
substantial spike in traffic for the Home & Realty 
sector. Hotels.com watched its audience levels 
soar, due in part to a high profile media blitz that 
enticed consumers searching for travel 
accommodations online. The reality hit Big Brother 
3 generated a surge in traffic for CBS, as captivated 
admirers snooped for additional content on the 
remaining members of the house. Broadband 
penetration continued its ascent, inching upward 
0.8 percent over last month; surpassing 26.4 
million active users (Nielsen/NetRatings, 2002). 


Such astonishing growth over a short period 
may be fairly viewed with scepticism by those 
charged with interpreting its commercial 
significance, but they would be unwise to be 
cynical about it, though many are. The fact is 
that there is already clear evidence that 
exposure to strong and readily available digital 
channels changes consumer behaviour in some 
key areas that relate to information gathering. 
Admittedly, the observed changes described 
here largely relate to consumer media 
consumption, but those academics who deny 
that generalist media consumption bears no 
relationship to more specialist information 
channels may be in for a rude awakening. 


Creating the promiscuous consumer 


The effect of using digital channels for 
information retrieval can be characterised as 
changing them from “users” to “consumers” 
and more particularly “promiscuous 
consumers”, whose somewhat promiscuous 
behaviour has been described by 
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David Nicholas and others as having “the 
makings of the biggest information bang ever 
(but some of us are deaf)” (Nicholas et al., 
2002) and more extensively by Bayler and 
Stoughton who describe digital consumers as 
“highly demanding, unsentimental, and 
generally unwilling to assign loyalty” (Bayler 
and Stoughton, 2002, p. 27). Bayler and 
Stoughton also argue persuasively (but 
worryingly for many publishers) that the 
“content is king” mantra we have all heard 
repeated so many times is not only erroneous, 
it is positively a barrier to establishing trust 
with digital consumers. They argue that 
“content is king” produces the effect of 
ignoring or even abusing the customer’s time 
by overloading them with irrelevant content 
in the goal-directed environment of the Web 
(Bayler and Stoughton, 2002, pp. 99-122). 
The loyalty shifts that Bayler and 
Stoughton identify are already being 
witnessed in consumer media. Figure 1 shows 
distinct shifts away from television and video 
watching, magazine reading and even sleep in 
those UK households where Internet access is 
established. The picture in the USA is not 
significantly different. Remember, too, that 
these shifts have largely taken place in a single 
decade, though the underlying trend in all 
consumer media since about 1950 has been a 
slow, steady adoption of electronic media at 
the expense of print media, particularly for 
news and information gathering. In the UK, 
one or two sub-trends, such as the spectacular 
success of the consumer magazine industry in 
the 1980s and the relative strength of regional 
newspaper publishing in the 1990s, have 
bucked the main trend. Overall, however, the 
move in consumer information gathering 
habits, especially among young people, has 


Figure 1 UK users ~ change in media consumption 
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been away from print and into electronic, and 
away from one or two trusted sources to the 
over-sampling of multiple sources. 

Some of the main changes in the pattern of 
consumer behaviour are being seen on the 
commercial side of digital delivery. They will 
also vote with their wallet or purse, as well as 
their information-gathering brain, as the 
astonishing growth of online air travel 
booking has shown, despite the effect of 9/11: 


According to Nielsen//NetRatings (NASDAQ: 
NTRT - news), the European airlines sector has 
more than doubled its online audience since last 
October, with 120 per cent growth in ten months 
... What’s interesting is that the fastest-growing 
sites — Lufthansa, BA and KLM — are all the sites 
of well-established non-budget airlines ... 
Consumers still want cheaper flights, as the 
success of travel portals shows, but they can get 
those flights now from a range of operators, not 
just “budget” airlines. 

This shows that the established airlines are more 
Web-savvy now, more competitive on price, as well 
as on reliability and customer care issues, enabling 
them to compete more effectively with the budget 
online airlines (Nielsen/NetRatings, 2002)[1]. 


The big question for the information 
profession is if “Today’s information 
consumer is a ‘flicker’ and flicks in response 
to massive consumer choice” (Nicholas et al., 
2002), how far will this behaviour extend to 
organised information retrieval? We have, 
after all, in our lifetimes, witnessed the 
transformation of “the reader” into “the user” 
in response to the first wave of mediated 
online activity. How far will the explosion of 
totally unmediated online and related activity 
go? What will be the impact of the mobile 
device, and particularly multimedia 
messaging systems (MMS), broadband 
wireless access and the integrated PDA/ 
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mobile? The evidence so far is that the 
answers are too difficult to call in these very 
early stages of the digital revolution, but there 
are early warning signs of the impact that 
digital can have on markets, producing 
difficulties for existing players. 

In developed consumer digital markets such 
as the UK, the top ten domains include only 
one pre-digital media brand, BBCi, and in both 
the UK and the USA the top ten online brands 
enjoy 80 per cent of the available advertisement 
spend. This was one of the major aspects of the 
failure of many Internet content providers to 
make money from the digital marketplace, as is 
clearly illustrated by the following exhibit 
(Figure 2) from a recent McKinsey Quarterly. 


Figure 2 First exhibit from McKinsey Quarterly 
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Conclusions 


The full power of the Web to 
disintermediate and to provide scale 
economies has yet to be seen, but it could be 
very significant. Fortunately, many of the 
scientific, technical and managerial (STM) 
publishers have been quicker to mount 
defensive strategies online than their 
consumer counterparts, and have faced 
fewer incursions from new players because 
of the inherent strength of their published 
archive. However, as McKinsey has shown, 
these publishers may not find that they 
enjoy the same margins and revenue from 
online subscriptions as they have from print. 
If the consumer information model is any 





Advertising revenue,! 


Total number of 


Number of competitiors 
controlling ~80% of 


$ billion competitors! advertising revenue! 
Network TV 7 4 
Cable TV 214 18 
Magazines 750 150 
Internet >9,000 10 





Note: 'US figures (2000) 


Source: Editor & Publisher: eMarketer: McCann-Erickson: National Cable and 
Telecommunications Association: McKinsey analysis 





Figure 3 Second exhibit from McKinsey Quarterly 
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guide, the advertising/subscription revenue 
ratio is somewhat different in the online 
market. Another exhibit (Figure 3) from the 
McKinsey Quarterly illustrates how much 
more exposed Internet content providers are 
to support from advertising revenue (which 
has in any case proved, as we have seen, 
much more elusive than for print 
publications) (Bughin et al., 2002). 

It would be presumptuous to claim, 
especially at this early stage of digital 
adoption, that the hard lessons learned in 
taking consumer information products 
online are directly applicable to specialist 
publishing markets such as STM. The fact 
remains that huge change is on the way for 
information providers of all kinds, and we 
have hardly begun to see the effects of it yet. 
We cannot dismiss the early evidence of 
changes in revenue portfolios and 
particularly in the way that consumers 
approach information online. 
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Note 


1 “Lufthansa tops Ryanair on the Web”, http:// 
uk. biz.yahoo.com/020930/21 8/darb4.html, 
(accessed 8 October 2002). 
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Introduction 


We were surprised at the initial hostility 
shown towards the first “end-user” recruits in 
the late 1970s/early 1980s and were always 
convinced that the disparaging of their search 
styles and information seeking behaviour was 
much to do with professional insecurity. 
However, all of this concerned bibliographic 
systems, largely devised for information 
professionals, academics and scientists. These 
were niche players, using niche and targeted 
systems, All that has now changed. Towards 
the end of the 1980s the number of end-users 
went through the roof, from hundreds to 
thousands and then on to millions in the 
1990s and tens of millions in the new 
millennium. No one disparages them 
anymore — there are too many of them for 
that; they have considerable clout and we now 
call them consumers, as recognition of their 
new found (economic) powers. 

It was not just the numbers of people using 
the systems that changed so dramatically, the 
types of people using the systems changed 
dramatically too: people without any previous 
experience of information systems — children, 
the elderly, those who had never been to 
university, and the unemployed. Of course, 
the systems themselves were changing fast. 
No longer OPACS to find references, but 
now the Web to download music or shop, or 
digital interactive television (DiTV) to check 
out on an ailment with a broadband nurse. 
No longer static, archival systems in public or 
work places, but mobile systems, real-time 
systems, interactive systems in the home; 
systems that owed more to the media world 
than they did to the world of information 
retrieval and references. Moreover, systems 
that were very cheap to use compared to their 
predecessors. 

The end-user has become the information 
consumer, indeed, the information player. 
Player is a good term that recognises that 
today’s information seeking can be 
interactive, recreational, social and 
competitive and acknowledges the 
characteristics of individuality and activity 
that define today’s online engagements 
(Nicholas and Dobrowolski, 2000). 

Given the seismic shifts chronicled above 
one would logically expect the new digital 
information consumers to act and react very 
differently from those landmark Dialog, Nexis 
and Reuters end-users. But there are not the 
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data to tell us how differently. Information 
researchers still seem obsessed with students 
and academics — easy to research groups, but 
hardly unknown, representative or even 
strategic. Most worryingly perhaps, 
unrepresentative, old and dangerously 
obsolescent models of information consumer 
behaviour are being peddled around. 


Aims and objectives 


The aims of this paper are to provide early 
intelligence as to how the new digital 
information consumer behaves when seeking 
information, advice and services online, 
whether this is via the Web, digital interactive 
television or touch-screen kiosk. It also aims 
to show how the information fingerprints, the 
first footfalls of millions, of people can be 
charted and evaluated. We now have more 
accurate ways of mapping the new 
information landscape, which possibly brings 
into question the small and sketchy pictures 
we drew of the “old” landscape. 

And it is precisely what these new 
information consumers are getting up to and 
what they may do next which concerns us 
most here. We have been amongst the first 
people to study one of the first — and much 
hyped — “flights” of consumers to the Web: 
that of newspaper readers (Nicholas et al., 
2000). Millions of people flocked to the 
newspapers we studied — The Times, Sunday 
Times, The Independent and The Guardian. 
Here we obtained our first glimpses of the 
“upside down” new world order of the digital 
consumer — all these sites were 
haemorrhaging money ~ as they are still. The 
digital consumer does not like paying directly 
for information and the advertisers that prop- 
up these sites are not at all convinced that 
consumers read the advertising on these sites 
because they do not stay around long enough. 
Despite the digital largesse of the media 
magnates, people still seem happy to buy 
hard-copy newspapers. In consequence, their 
hard-copy cousins have to bail them out, 
something that is now impossible to do given 
the shortfalls in advertising income resulting 
from the events of 11 September. 

More recently (2000-2003) we have been 
charged by the Department of Health to 
evaluate the similarly much-hyped roll-out of 
digital health services to the nation (Nicholas 
et al., 2002a). Health, forgive the pun, is 
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allegedly the killer digital application and- 
large sums of Government money are being 
spent to ensure that is the case — the same, of 
course, was said about news (Nicholas and 
Frossling, 1996). The lesson that the digital 
consumer does not like paying for their 
information has been learnt (but not by 
everybody of course — see FT.com’s latest 
effort to get people to pay), but the question 
now is whether the Government has deep 
enough pockets to bankroll them? This time 
the roll-out is on a different and much more 
ambitious scale, the consumer is being offered 
three digital platforms from which to make 
their choice — touch-screen kiosk, Web and 
digital interactive television. Part of our work 
is to determine whether, for the consumer, 
this really represents choice or the digital fog 
descending. 

And most recently (2002-2003) we have 
been studying the use of mobile phones by 
teenagers — surely the most passionate 
communications love affair of all (Nicholas 
and Chivhanga, 2002). 

The digital consumer is rapidly 
metamorphosing into the digital purchaser. 
They now add a growing commercial 
consideration to the decision mix. Following 
Christmas 2001, 78 per cent of consumers 
stated that they would probably look to 
purchase “more” or “significantly more” 
online in Christmas 2002 (eDigitalResearch. 
com, 2001). In addition users’ expectations 
for Internet purchases are rapidly changing. 
They expect better pricing, ease of use, secure 
payments as well as the complete customer 
service back up they would get from 
traditional channels. Interestingly, and 
perhaps more worryingly for manufacturers 
and High Street retailers with large marketing 
budgets, “brand” and “high street” presence 
seemed to have little relevance to the digital 
consumer (eDigitalResearch.com, 2002) — an 
important characteristic of the digital 
consumer we shall return to. 


Methods 


Plainly a few people in Clerkenwell (City 
University) evaluating the information 
seeking behaviour of millions of people on 
digital information platforms as diverse as 
DiTV, the Web, touch-screen and mobile 
phone require a special type of methodology, 
and we believe we have that. All these systems 


Digital information consumers, players and purchasers 


Aslib Proceedings: New Information Perspectives 





David Nicholas et al. 


record the information activities of their users 
(transactions) as a kind of digital fingerprint. 
They do this on a real-time and continuous 
basis. We call them logs — they constitute the 
CCTV of cyberspace. Logs provide a 
methodological level playing field, which 
enables comparisons to be made between 
sites, services and digital platforms. They also 
enable us to pen information portraits of 
whole populations of people rather than small 
communities — although, of course, we can do 
this as well. We have logs for millions of 
health information consumers, in some cases 
for a period of as much as five years. As we 
write, thousands and thousands more digital 
fingerprints enter the database and will 
continue to do so for a number of years to 
come, so we can monitor change and obtain a 
video, rather than snapshot, of the rapidly 
unfolding digital revolution. The sheer 
numbers of consumers allow an accuracy of 
opinion and evaluation not dreamt of, 
utilising traditional research techniques where 
costs of collection are a direct (increasing) 
proportion to the number of users 
interviewed. New digital techniques of 
collection and analysis allow for a previously 
unheard of theoretical 100 per cent collection 
rate amongst the total user population of any 
particular digital service. This digital 
information tide places big question marks 
against portraits drawn on the basis of small 
and unrepresentative samples. In other words 
we might not only have been working on the 
wrong models, but on small and 
unrepresentative data sets. 

The logs we work with are anonymised and 
generalised but nevertheless tell us much 
about patterns of information seeking 
behaviour and a certain amount about the 
personal characteristics of the people 
searching the systems — age, gender, and 
postcode obtained through people willingly 
entering this data at the touch-screen kiosk or 
via a questionnaire. But not so revealing as to 
break privacy laws. For fuller details on the 
process of log analysis see Nicholas ez al. 
(1999). 

Used together with online pop-up and 
traditional questionnaires and interviews the 
data is particularly rich and revealing. It can 
be triangulated to give us a real “grip” of what 
is going on. Indeed, the data collection rates, 
as well as the quality obtained from 
unsolicited Web based research, seems to 
indicate that users have a propensity to 
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provide useful feedback of their experiences. 
Digital research methods allow instant and 
real-time feedback of opinion and habits from 
the digital consumer. 


Digital consumer characteristics 


Enormous and unprecedented numbers 
If we can believe every Web site “hits” release, 
digital consumer traffic is only ever measured 
in millions. Indeed, even if these counts are 
inflated and misleading, their sheer numbers 
are probably the digital consumers’ key 
identifying characteristic. The numbers are 
now so large that they have become 
meaningless — we expect them to be large, we 
get blasé. But there is a negative aspect to the 
impressive audience figures and their 
relentless growth — it puts people off analysing 
these large sets of data and thus they fail to 
obtain an understanding of the new digital 
information consumer. Even in the newly 
emerging digital health field that we have 
recently entered the numbers are beginning to 
rattle-up (Nicholas et al., 2002a). The NHS 
Direct Online Web site attracts around 1,500 
visitors a day. The 80 or so touch-screen 
kiosks of InTouch With Health attract going 
on to a 1,000 people a day; 400 people a day 
accessed the pioneering Living Health digital 
interactive television channel in the 
Birmingham area and 50 people send e-mail 
enquiries to NHS Direct Online everyday. 
And this is from a standing start. It is very 
early days indeed for digital health services. It 
is the reach figures that are really impressive 
though, especially in the health field. In the 
Birmingham area where a DiTV health 
service was rolled out recently, after six 
months just under 40 per cent of Telewest 
subscribers had availed themselves of the 
service. Of course, big numbers make big 
waves. 


All-powerful 

There has been a massive shift in power from 
information provider to information 
consumer. Information players have 
enormous power. And this is not just because 
of their numbers. It is also to do with the fact 
that they have huge choice (something we will. 
pick-up later on) and the backing of 
governments everywhere. Providing power 
and choice to the consumer has become a 
political mantra. 
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Personal characteristics 
Digital information users are no longer just 
academics or professionals. Nor are they just 
adults or young IT obsessed men. In fact our 
health research shows that there are quite 
different audiences for the various digital 
platforms and services (Nicholas et al., 
2002a). Thus DiTV users tend to be older 
(well over a third were aged over 55); health 
Web site users were typically 35-54, and kiosk 
users were largely young (under 35 year olds). 
Women were slightly more likely to use digital 
health services than men, except in the case of 
DiTV and, interestingly, a kiosk located near 
the pharmacy in Safeway! DiTV users were 
also more likely to be unemployed, a 
housewife/mother, or retired. To the DoH 
this was encouraging in that it supported the 
argument that DiTV throws an ICT lifeline to 
those who have traditionally been excluded 
from the digital revolution — the poor and 
socially excluded. And we all know about how 
teenagers are attracted to mobile phones. 
The Internet, initially a male dominated 
domain, now reflects the gender balance of 
the general population. In addition we 
perhaps cannot rely on the traditional 
demographic profiling used to neatly predict 
consumer behaviour. There is a need to revisit 
how users view digital media and how these 
new user profiles react to the fast moving 
delivery methods activated by this media. 
OPAC and commercial online researchers 
never had to contend with one kind of “user” 
found in large numbers in cyberspace — the 
robot or the spider, sent out, for instance, by 
search engines for indexing purposes. Robots 
were found to account for 30 per cent of users 
in our newspaper studies (Nicholas et al., 
2000). The question is do they constitute 
users and should they be counted with human 
users? Most software programs tend to 
exclude them, but that is too easy. 


Promiscuous users — short attention 
spans and short visits 

Today’s information consumer is a “flicker” 
or a “bouncer”. As children use the remote to 
channel hop, so their parents information hop 
or bounce their way across the digital 
information terrain, and especially the Web. 
Our studies show that one third of kiosk users 
never really get beyond the initial menu pages, 
the figure for DiTV is a quarter and for the 
Web, where it is much easier to bounce 
around because of the greater choices on 
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offer, around 40 per cent. We call these 

people “bouncers”. We are all bouncers. This 

results from a number of factors: 

* having very short attention spans; 

* running up continuously against home 
pages we do not like the look of; 

* being hostage to a retrieval system (the 
search engine) that is constantly coming 
up with empty, irrelevant or uninteresting 
postings; 

* downloading and printing of data for 
“use” later; 

* people find surfing a stimulating leisure 
activity, like window-shopping (it is 
connectivity rather than interactivity that 
is sought); and 

* huge choice. 


Even those who penetrate the sites — rarely go 
beyond the home page or wander very far. 
They nibble rather than bite. This kind of 
behaviour costs the information providers and 
sponsors a lot of money. They would much 
prefer we hang around, that the sites be 
“sticky” and more importantly come back for 
more. 


Promiscuous 

The most important reason why people flick 
or bounce is because they have choice — the 
wealth of alternative digital sources enables 
them to do so. Data suggest that we visit 
many sites but “use” few (eDigitalResearch. 
com, 2002). We are “promiscuous” 
consumers. In addition users are more aware 
of their ability and come to realise that they 
are empowered to cross traditional 
boundaries, whether these be geographic (e.g. 
wow.com shipping CDs to UK from Hong 
Kong, offshore gambling) or knowledge based 
(health issues). In research into Web site use 
we have found that users visit many sites but 
currently purchase from few sites. 

What we do not know is what arises when 
previously niche and specialist services, like 
health services, are now juxtaposed with 
holiday, football and retail sites/services in the 
digital information window that is DiTV or 
the Internet. Plainly there must be some kind 
of a knock-on or interaction, but as yet we can 
only guess as to what it might be. 


Untrusting 

The multiplicity of access points to 
information — TV, newspapers, consumer 
magazines, learned journals, radio, Internet, 
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PDA, Online services — confuses the 
consumer and leads to a breakdown in trust 
and information dissonance (Bayler and 
Stoughton, 2002). In comparison with print 
media, all electronic media (including 
analogue broadcast) are difficult to navigate 
and reference. In essence they are two- 
dimensional, whereas print is three- 
dimensional because the shape of the whole 
experience can be successfully deduced from 
the physical form. For example, even the 
briefest knowledge of a print newspaper 
(which is not usually indexed at primary use 
level) leads the consumer to an understanding 
of what they will find and where in a relatively 
standard formula. Digital formats, and the 
Web in particular, have a tendency to break 
down this easy familiarity and leave the 
consumer floating in a sea of uncertainty. The 
consumer responds by demoting 
considerations of trust and loyalty. 

Of course, there are also the “authority” 
problems that arise from a relatively new and 
fast changing environment with new players 
coming in all the time. Authority is plainly 
“up for grabs”. It is difficult to determine 
ownership because there are so many parties 
associated with the production of a digital 
information service. Take a touch-screen 
health information kiosk, produced by 
Surgerydoor, containing some information 
from NHS Direct Online and located in a 
Safeway store. To the person shopping in the 
supermarket who sees the kiosk and decides 
to use it, whom do they attribute its qualities 
to: Safeway, Surgerydoor — a company of 
which they have probably never heard, or to 
the NHS, should they be alert enough to spot 
its logo. There is no easy answer of course, 
but one suspects they might say Safeway. 

At one time consumers trusted in the 
experience of the information provider. In the 
flattened world of the Internet, where all 
things are seemingly equal, experience 
appears less important than ever — hence the 
diminishing role of information professionals 
in the search process. 

Our research shows that where advertising 
is obtrusive there was a reduction in 
trustworthiness and that traditional quality 
marks — like that of the NHS - are of limited 
worth (Nicholas et al., 2003). Half of the 
respondents in one study we conducted did 
not even notice that the material was 
produced by the NHS (despite the 
strategically placed logo) and in another study 
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younger cable respondents were less likely to 
recognise the NHS as a symbol of trust than 
their elders. 

There is a sense that consumers are more 
brand conscious, but less brand loyal. This 
applies to information just as much as 
whiskies. Information specialists have largely 
accepted that information has become a 
commodity (sometimes, but sometimes not, 
with an expressed value attached to it) but do 
not often accept the consequences of this. 

Given all this, how will the digital consumer 
make “purchase” decisions in the future? This 
is a question that will continue to exercise the 
commercial mind over the coming years. We 
have already seen fundamental changes to 
purchase patterns in some market places, 
causing dramatic shifts in corporate thinking, 
e.g. books, music, low cost air travel. As the 
search and purchase habits of the digital 
consumer continue to change and evolve — 
questions such as brand loyalty and brand and 
product identification become increasingly 
important. For example, traditionally a 
consumer shopping for groceries uses a 
variety of complex processes in making 
purchase decisions. How will they make those 
same decisions when faced with possibly more 
information, more choice and a different set 
of purchase influences? How does a brand 
aware consumer find and then successfully 
purchase the product he or she wants to buy 
without being seduced by a lower cost 
digitally available alternative? We have 
already seen some traditional distribution 
channels forced to mount successful defences 
of their income streams. Free music 
downloads through Napster.com vs 
traditional music publishers being one 
example. How many more will follow? 


Novice retrievers 

The digital consumer is faced frontally with 
the problem of retrieval. It is an inescapable 
process of digital information consumption. 
For all but experienced researchers (a 
minority of users), this is the first time they 
have faced the dilemmas of precision versus 
recall, of noise, and of deciding on the validity 
of sources. Information providers have to be 
aware of this and solve these problems in the 
same way as, in the past, they have provided 
contents pages, indexes and TV guides. The 
role of content is becoming inescapably 
linked, in the user’s mind, with its ease or 
difficulty of retrieval, and this is often 
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something quite outside the control of the 
original publisher. Publishers therefore are 
moving beyond questions of “how accurate 
is this?” or “how readable?” to “how visible 
is it?” 

The concept of an information player is a 
very helpful one in understanding how the 
general public interact with digital systems — 
and, also, re-visiting what we have said and 
learnt about them in the past. Using a sports 
analogy, few goals are probably scored in a 
textbook manner. Thus the stars of the 
football world, like Paulo Di Canio, are 
famous for their ability to do the 
unconventional, the unusual, the creative. In 
any game of football players do a lot of things 
that are not in the training/coaching manuals, 
but plainly they have received training. 
Similarly, a lot of information is collected by 
unconventional, unusual or serendipitous 
means. Thus, maybe, what is seen as 
minimalist and idiosyncratic information 
behaviour is not so odd, strange etc. after all — 
maybe it was just creative. Certainly this kind 
of behaviour is more in tune with the thinking 
of Google than Dialog.’ 

Amongst the consumer there is almost a 
total delegation of trust to the search engine, 
which is invested in almost magical powers of 
retrieval. So powerful is the magic that few 
people even ask how the search is conducted, 
what is being searched (and, more 
importantly, what is not). Now, arguably, this 
would not be too serious if all people were 
doing was searching for references or 
e-shopping, but when it is your health or 
welfare that is a different story (and this is not 
fully realised). 


Want it now 

Total access and speed of delivery appear to 
be the consumers’ key information needs 
requirements. They have certainly surpassed 
traditional quality concerns. which is 
disturbing for information professionals, who 
have traditionally played the role of quality 
guardians — and still cling on to the role with 
suggestions of portals and the like. Total 
access and speed of delivery are the perceived 
characteristics of Amazon.com, but not of the 
academic or public library. And, of course, 
this is why the mobile phone has proved such 
a spectacular success — it is an always on, real- 
time system. Nobody today wants to wait; 
nobody wants to queue — even if they could. 
Time plainly is a rare commodity. 
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Connectivity is what our teenage mobile 
phone users tell us they yearn for. And what 
will surely drive the use of the mobile phone 
as an information retrieval medium will be the 
vast and ever-increasing amounts of real-time 
information becoming available. 

Real-time information, once the exclusive 
and treasured domain of journalist/ 
stockbroker, is open to all and is now what 
everyone wants, it is the benchmark. 


Consumers converging? 
Convergence is the buzz phrase, however, 
everything that has happened in recent years 
seems to suggest the very opposite is 
happening. DiTV is the new convergent kid 
on the block. DiTV, the vehicle for WebTV, 
is not only a brand new platform, about which 
very little is known, it is also a platform on 
which much hope is invested. There is a 
sense, amongst politicians certainly, that this 
is the platform. It has real consumer reach, 
genuine general public credentials and will 
thus be the main conduit for a whole range of 
e-government and e-local government 
services for the public. Essentially, the 
argument goes, everybody has a television 
(and soon a digital television), more than one 
in many cases, everybody is familiar with 
them, and maybe most importantly it throws 
an ICT lifeline to all those who have been 
excluded from the digital revolution to date — 
the poor, socially excluded, elderly etc. 
However, our data tells us that it is very 
much digital horses for courses. Web users 
certainly are not convinced at all about the 
benefits of DiTV. Platforms are one further 
manifestation of choice. 


Interactive consumers? 

Interactivity — surely the great sell and the big 
disappointment? Interactivity must be the 
most abused, misused word in the digital 
world. So much so we have introduced 
another word, transaction, to differentiate 
services that offer more than the ability to 
interrogate a database and download a few 
pages of information. Transactional services 
are those services that involve a much closer 
engagement, a much greater degree of two- 
way interactivity. Many e-services involve 
nothing more than digital leaflet distribution 
or a mail-order facility. However, in the case 
of one of the digital introductions we have 
evaluated — the In-Vision service, a 
broadband on your television, in your home, 
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nursing service — you have something that 
begins to meet the consumers’ expectations of 
what interactive services are all about 
(Nicholas et al., 2002b). Interestingly, 
whether it was too intrusive or just too novel, 
the take-up was disappointingly low (130 
people used the service in six months). 

In addition there are equally strong signs 
that the digital consumer will research and 
take commercial decisions online when 
presented with a viable alternative and less 
expensive method of decision making and 
subsequent acquisition. 


Unpredictability 

If nothing else, the digital consumer is 

unpredictable. Of course this unpredictability 

partially stems from the fact that the digital 
information consumer is such a recent and 
fast emerging phenomenon. 

When a digital service is rolled out the 
unexpected inevitably happens, take: 

*  Touch-screen information kiosks. Health 
touch-screen information kiosks, 
designed with the elderly in mind, 
turned out to be most popular with the 
under 15s. 

* Mobile phones. Few people anticipated the 
phenomenal rise of the mobile phone. 
And no one at all surely anticipated that it 
would be children who would fuel much 
of this growth. After all, the cellular 
phone was originally conceived as a 
communications technology for people 
away from the office — farmers and sales 
executives. Then there is the enormous 
popularity of text messaging amongst 
children, on a medium that was designed 
to trumpet the well-known attractions of 
oral communication. 

* Digital libraries. The Big Deal was a 
solution to the “serials crisis” which was 
developed during 1997. The basic deal is 
that the publisher continues to receive the 
same amount of money as they received 
from print subscriptions from all the 
libraries in a consortium. For a small 
surcharge, all the libraries in the 
consortium receive electronic access to all 
the journals published by a particular 
company. The advantages of the deal 
were seen as: increasing content delivered 
per dollar spent; stabilising economic 
relationships; maximising content 
delivery per publisher; maximising use via 
electronic access. All these things were 
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achieved but what was not anticipated 
was that journals previously not 
subscribed to in print were used more 
than those previously subscribed to — 
questioning as a result intermediary 
involvement in the selection process. Of 
course, the unbundling of articles from 
their journals that occurs in the digital 
environment could well provide the 
explanation (Nicholas er al., 2003). 

+ E-commerce. New brands and distribution 
methods can earn the trust of consumers 
and become leaders in their field in a very 
short time span, challenging traditional 
methods of distribution (e.g. amazon. 
com, easyJet.com). 


Outcomes and benefits 

What happens when the controls are taken off 
and people who have been starved of 
information, or told only what information is 
considered good for them, wake up to the fact 
that there has been a revolution and now 
information is everywhere and easily 
accessed? Flood the area with information — 
pipe it into people’s homes, surgeries, railway 
stations and libraries and something really big 
is likely to happen. We have the makings of 
the biggest information bang ever heard. The 
first rumblings can already be heard — after all 
there are millions of consumers on the 
information road now. 

Performance measurements have been 
important to information and computer 
professionals, but usually only in terms of 
search engine efficiency and the relevance 
judgements of the users — before they actually 
go away and digest and make something of 
the information. But of course the real 
concern now is that, when life-critical 
services, like health, are moving over to the 
digital environment what do consumers do 
with the information and how does it impact 
on other services and aspects of their lives. 
The fact that someone used a Web site and 
maybe consumed the information or bought 
the product is interesting, but not half as 
interesting as what it did for them and how it 
affected the other things they do. 

There is more than a little suspicion that the 
Government believes that the provision of 
digital information direct to the consumer will 
go some way to reducing the pressures on the 
health service — a case, maybe, of patient heal 
yourself. There is indeed some evidence to 
suggest this is happening. One third of Living 
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Heaith television channel viewers said that the 
information found either helped or helped a 
lot in improving their condition. This 
constitutes an important use of a health 
information service. The younger the 
respondent the more likely they were to report 
substituting information they had found for a 
visit to the doctor. There were signs too of 
joined-up health information acquisition. 
Well over half DiTV users queried the service 
for information about their consultation with 
the doctor either before, after or before and 
after their consultation. 

In commercial markets we also see 
changing purchase patterns as information 
becomes more readily available and 
consumers understand how to find it. 
Consumers have steadily increasing access to 
more information to pricing, availability and 
details of products and services. Despite the 
effect of 11 September the low cost airlines 
report increasing volumes of travellers 
(easyJet up 36 per cent half year on 
corresponding half year) with bookings over 
the Internet taking an increasing proportion 
of bookings (easyJet, 2002). 


Reflections and conclusions 


Tt is a story of big change, all change and rapid 

change. There are lessons for us all: 

* Most of all that the behaviour of the digital 
consumer needs to be tracked and 
evaluated as a matter of course. We have 
shown how spectacular and unexpected it 
can be. As more services go digital we need 
to fill the intelligence vacuum quickly. 

+ Information vendors and providers 
(libraries included) will have to engage in 
the same processes as all other providers 
of goods and services, and put customer 
relationship management (CRM) at the 
centre of the information experience 
rather than at its periphery (or 
commonly, nowhere at all). As Rakesh 
Sood (2002) pointed out in an article 
published in TheStreet.com: 


What does the emerging promiscuous era hold 
for vendors? Locking in customers becomes 
more difficult. Vendors will need to further 
differentiate their offerings, re-examine their 
value proposition and stay on their toes while 
executing at a higher level on a continuous basis. 
While periods of temporary dislocation are 
inevitable, the customer is the ultimate 
beneficiary. Is it likely that, before too long, the 
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adage “the customer is king” becomes 
transformed into “the promiscuous customer is 
king”? ; 


* Not only do we need to challenge our 


assumptions about the information 
consumer, but we also need to ask where 
digital information provision, and 
especially the Web, is taking the 
consumer. Consumers are being fast- 
forwarded to a data-driven world, in 
which the only thing that changes is the 
amount of information made available to 
them and the platforms on which this 
information is disseminated. Loads of 
links, endless links, thousands of postings 
characterise the journey — an endless 
information journey with seemingly no 
destination. Consumers are unlikely to 
even remember where they went on their 
journey; they will retain little of the 
information, or how they can return to it. 
There is no sense of knowledge building. 
People are gradually being divorced from 
the roots of communication/knowledge. 
The problem is that the (false) sense of 
engagement and action associated with 
online searching/surfing makes the 
consumer feel that something is being 
achieved, but this is not always the case. 

¢ The other worrying phenomenon, and it is 
linked to the above bullet, is that it is not 
the information or content providers that 
are making anything out of digital 
information provision but the telecoms 
companies. Telecoms companies are 
technologically driven and think about 
nothing more than backbones (and 
profits). If things continue like this for long 
there could soon be no content providers 
worth the name. Could you imagine a 
world in which supermarkets gave their 
produce away for free and the councils 
charged you for access to the roads that 
led to the supermarkets? No, well that is 
the situation we have in cyberspace. 
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Biomedical research evaluation has traditionally been 
based on analysis of outputs and their citations by other 
papers. However we should try to map the routes by 
which research actually improves patient care and reduces 
illness, and develop indicators for them. We must allow 
for the lengthy time-scales involved and the importance of 
researchers being physically close to healthcare 
professionals, whose practice can be improved through 
international and governmental regulations and through 
approved guidelines. Each of these will depend on a body 
of research evidence. We must also evaluate the effects of 
research on policy makers and the public, who often learn 
about it through the World Wide Web and through the 
mass media, particularly newspapers. The latter provide a 
major bibliometric resource but one that needs to be 
tapped in individual countries using common standards in 
order to provide internationally-comparable indicators. 
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Introduction 


The evaluation of research is one of the tasks 
to which information scientists have been able 
to contribute through their analysis of 
databases of publications. They have thus been 
able to help shape the research agenda, 
especially in biomedicine where articles in the 
serial literature form the standard currency of 
research activity (Lee, 1998; Research 
Fortnight, 2000). There are several databases 
that list these articles, such as EMBASE, 
MedLine and the Science Citation Index 
(SCD. They can be interrogated in various 
ways in order to identify the outputs of 
selected countries or research groups, or 
papers in particular subject areas, or both, and 
then to compare these sets of papers on 
different criteria. Typically, numbers of papers 
are counted and compared with the inputs to 
the research (money or people) (Leta et al., 
1998; Noyons et al., 1998). Measures of 
impact have also been developed, mostly 
dependent on the frequency with which the 
papers are referenced or cited (Garfield and 
Welljams-Dorof, 1992; Bourke and Butler, 
1996; Kostoff, 1998; So, 1998; Meho and 
Sonnenwald, 2000) by other researchers. 
These indicators can be determined from the 
SCI, or from journal-based measures such as 
the mean citation counts for their papers 
(Balaban, 1996; Lewison, 1998) or their 
subjective esteem by researchers (Lewison, 
2002a). In recent years, citations by patents 
have also been analysed to provide an indicator 
of technological utility (Narin, 1994; Narin 
and Olivastro, 1998), but this is less useful as 
very few papers, typically only 2 per cent in 
biomedicine, are ever cited by a patent. 

However these measures do not go far to 
show whether the research is actually having 
an impact on the real world outside the 
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laboratory. We want to know if human health 
has actually benefited from the research, 
either through better patient care or from less 
illness. Figure 1 shows some of the numerous 
and rather tortuous links by which biomedical 
research may have such an effect. It makes 
two main points: first, the links involve several 
steps and much time and, second, 
government policy is clearly critical to the 
whole picture. This is especially so in a 
country such as the UK with a state-funded 
National Health Service (NHS) and a 
tendency to bring in lots of regulations 
influencing people’s lifestyle choices. 
Examples are the requirements to wear seat 
belts in cars and helmets on motorbikes, and 
bans on smoking in public buildings and on 
public transport[1]. 

This paper seeks to describe some new 
ways in which parts of this diagram can be 
mapped. They will be accompanied by 
examples, which give an idea of the main 
parameters that can be analysed, and some 
initial results. These are not definitive, but 
they do suggest that conventional, citation- 
based, measures of research utility may be 
very partial and in need of supplementation 
with quite different indicators if the true 
impact of biomedical research is to be 
appreciated. In particular, it appears that 
although citations may be appropriate for 
basic research, they can under-evaluate 


Figure 1 The links between research and better health 
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clinical work. Moreover, the routes by which 

research influences health are primarily 

national ones, and therefore it is necessary 
for bibliometricians (as information 
scientists working in this field call 
themselves) to develop their own national 
indicators rather than rely on international 
citation data. 

If one is analysing the impact of research in 

a small country, one should perhaps start by 

asking the question, why should that country 

carry out biomedical research anyway? After 
all, it is very unlikely that key discoveries will 
be made there compared with the larger and 
better-funded labs in the USA. Small 
countries nevertheless do carry out their own 
biomedical research for a number of good 
reasons, amongst which we can list: 

* to improve the training of doctors and 
people working in the professions allied to 
medicine (e.g. dieticians, nurses, 
physiotherapists, speech and language 
therapists); 

* to absorb research conducted elsewhere 
and be able to apply it quickly in the 
country; 

e to foster an intellectually well-developed 
healthcare culture that will attract able 
physicians and surgeons who can provide 
a high standard of patient care; and 

* to research diseases of local and national 
significance but which may be of marginal 
importance internationally. 
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It is not surprising, therefore, that many 

countries will want to evaluate their research 

by reference to national indicators of 

effectiveness rather than the international 

standard of citations from scientific papers. 

In this paper we consider the development 

of three sets of indicators, which can be of 

national or international origin, and can 

reveal some ways in which biomedical 

research can affect health: 

* through standards limiting human 
exposure to pollutants; 

* through the development of clinical 
guidelines for patient treatment; and 

* through the reporting of biomedical 
research in the mass media, specifically in 
newspapers. 


The effects of pollution on human health have 
been well-established: the main effect in 
developed countries is now from air pollution 
(Hahn, 1996; Konisky, 1999), particularly 
from small particles emitted from vehicles, 
given that smoke and SO, from most 
stationary sources have been drastically 
reduced over the last half-century. However 
there is popular concern over the safety of food 
(Birchard, 1999; Gregoriadis, 1999; Boseley, 
1999) and the dangers of pesticide residues 
have been the subject of books and other 
reports (Colborn et al., 1996; O’Brien, 1996; 
Kondro, 2000). Their true overall contribution 
to ill-health seems likely to be almost 
vanishingly small (Doll and Peto, 1981; 
Gough, 1989; McGinnis and Foege, 1993), 
but in the public’s mind they are things that 
ought to be removed. There are, indeed, 
international standards for the levels of 
permitted residues of various pesticides in 
different foods, and they are developed by a 
United Nations body, the Codex Alimentarius 
Commission (CAC), which was formed jointly 
by the Food and Agriculture Organization 
(FAO) and the World Health Organization 
(WHO) in 1962. We have looked at a sample 
of the evaluations produced by the CAC and 
the references in them in order to see which 
countries’ research is used to develop these 
standards, how long it takes for papers to be 
cited and what type of research is used. 

The second area of study is clinical 
guidelines. In the UK, there are many 
different sources of guidelines but they do not 
all enjoy widespread dissemination or 
command the same degree of respect. Until 
fairly recently, guidelines assessed by the 
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NHS Appraisal Centre for Clinical 
Guidelines at St George’s Hospital Medical 
School had a superior status. In a study of the 
references listed in the SCI in a sample of 15 
of them Grant and his colleagues (Grant et 
al., 2000) found that: 

* British research was cited about 2.5 times 
as much as expected from its presence in 
the world biomedical literature (10 per 
cent); 

* most of the references were to papers in 
journals classed as “clinical” rather than 
basic; 

* the median time lag between publication 
of papers and citation in guidelines was 
eight years. 


Since then, the National Institute for Clinical 
Excellence (NICE) has started developing 
guidelines for the NHS and, to date, 47 
guidelines, supported by detailed Technology 
Assessments, have been published. The 
references in these documents, many of which 
deal with the efficacy of pharmaceutical drugs 
for particular disorders and diseases, have 
been analysed in order to up-date Grant’s 
study and reveal which papers have had an 
influence on patient treatment in this way. 

The third area of study is the citation of 
biomedical research news in newspapers. 
Although it can be argued that television may 
have more influence, it is much harder to 
analyse the references to biomedicine in that 
medium because of the difficulties in 
monitoring them and because the references 
will vary so much in extent and type. 
Newspapers have a wide readership, from 
politicians and senior officials to healthcare 
administrators, doctors, researchers and the 
general public, many of whom are becoming 
very health-conscious. Results are presented 
from a two-month sample of UK national 
newspapers, more to show the types of 
questions that can be asked and answered 
than to give definitive statistics. There is also 
discussion on how such a project could be 
internationalised. 


Pollution control: pesticide residues 
in food 


This project was carried out by Potenza 
Atiogbe as part of her work for a MSc degree 
at City University (Atiogbe, 2001). It began 
with the identification of some 32 individual 
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pesticides, which had been subject to 
toxicological evaluation, some more than 
once. Table I shows the ones most subject to 
evaluation. 

The 74 individual evaluations were written 
from 1976 to 1995 and covered several 
aspects of pesticide residues in food: 

* technical problems in the measurement of 
(often) small concentrations of pesticides; 

* dietary surveys to establish how much of 
the given food is eaten both routinely and 
by some special groups; 

* toxicology studies to show whether the 
ingestion of given amounts of pesticide is 
likely to be harmful. 


The CAC evaluations and the references in 
them were taken from the Web, and the first 
1,000 individual references were copied and 
pasted into an Excel spreadsheet so that they 
could be looked up in London libraries. Some 
740 of them were found and their addresses 
and funding details were noted, and entered 
back on to the spreadsheet for analysis. In 
particular, the differences in time between the 
date of publication of the papers and their 
citation in a pesticide evaluation were noted. 
It was necessary to have a comparator file so 
that the literature cited by the guidelines 
could be seen in context. This was developed 
by the creation of a pesticide “filter” called 
PESTIWLD. This took papers from the SCI 


Table I Pesticides frequently evaluated for human 
effects, 1975-1995, by the Codex Alimentarius 
Commission 


Pesticide 


Captan 
Carbendazim 
Fenitrothion 
Imazalil 
Methamidophos 
Acephate 
Carbaryl 
Chlordane 
Chlorothalonil 
Diphenylamine 
Fenamiphos 
Fenthion 
Lindane 
Malathion 
Paraquat 
Chlordimeform 
Dichlorvos 
Dimethoate 


NN NM ww Ww Ww Ww ww Www p hr HL MN 
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from 1991-1998 if they were in one of three 
specialist journals (F. Pesticide Sci., Pesticide 
Sci. and Pesticide Biochem. Physiol.) or had the 
name of a pesticide in their titles, and also had 
a title word indicating that the effects of the 
pesticides on humans or the environment 
were being studied. For example, assess* or 
carcinogen* or degrad* or expos* or pollut*, 
or cat or dog or mouse or rat, the most 
commonly used animals for testing, would 
indicate a paper testing the safety of a 
pesticide. This file contained 5,461 papers. 
The references from the pesticide toxicology 
papers published in 1998 were downloaded to 
form a fourth file with 19,034 papers and 
their dates were analysed so that the 
distribution of times between publication of a 
paper in a journal and its citation by another 
journal paper could be determined. Data were 
also obtained on the research levels (from 
clinical or applied to basic) and the potential 
impact category of the journals in which the 
CAC evaluation references and the pesticide 
toxicology papers were published. 

Figure 2 shows the comparison between the 
time differences between publication and 
citation for these two groups of papers. It is 
evident that it takes much longer for papers to 
be cited on CAC evaluations than on 
scientific papers; this result can be compared 
with that found (Grant et al., 2000) for 
clinical guidelines. Our conclusion is that it 
may take more than five years for the true 
utility of research papers to pesticide 
regulation to become apparent, even though it 
is possible to gain a good idea of their likely 
citation performance at the end of this time. 

Figure 3 shows the frequency with which 
papers from each of 20 countries are 
represented, both in the CAC evaluation 
citations and in the pesticide toxicology 
papers. It enables us to see whether the 
former reflects the relative contributions to 
the literature of these countries, as 
represented by the latter. It appears that the 
USA is over-represented and several 
countries, notably ES = Spain and to a lesser 
extent FR = France and JP = Japan, are 
under-represented. This reflects their much 
smaller relative contribution to pesticide 
toxicity literature in the 1970s and 1980s, 
from which most of the CAC references 
stemmed (60 per cent were from 1979 and 
earlier). For example, the percentage 
contributions of these three countries to the 
pesticide toxicity literature in 1981-1984 were 
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Figure 2 Comparison of the ages of references from CAC evaluations (CAC REFS) and those from a sample of 
pesticide toxicology papers in the SCI (PESTIWLD) 
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Figure 3 Comparison of the percentage frequency with which papers from various countries are cited on CAC 
pesticide evaluations (n = 740) and in pesticide toxicity papers in the SCI for 1995-1998. Log scales for both 
parameters 
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respectively only 1.0 per cent, 3.1 per cent Japanese papers in 1981-1982 were in 

and 10.1 per cent, compared with 7.9 per Japanese (in ¥. Pesticide Sct.) and the 

cent, 5.1 per cent and 11.7 per cent in 1995- proportion was probably higher in the 1970s. 
1998. The under-representation of Japan may However it is remarkable how fairly most 
also be due to language: 23 per cent of other countries, including some developing 


36 


Beyond outputs: new measures of biomedical research impact 
Grant Lewison 





ones (CN = China, EG = Egypt, IN = India) 
are represented in the CAC evaluation 
references. This suggests that science in these 
countries is by no means ignored in the 
international decision-making process for 
pollution recommendations. 

An analysis of the CAC evaluation 
references and the comparator pesticide 
toxicity papers by research level and by 
potential impact category is shown in Table II. 
Both sets are relatively basic, the CAC 
evaluation references slightly more so (the 
difference is statistically significant at p ~ 0.002 
per cent). Rather few of either set are in 
journals of high citation impact (the norm in 
biomedicine is 10 per cent in PIC4, 20 per 
cent in PIC3, 30 per cent in PIC2 and 40 per 
cent in PIC] journals). However the CAC 
references are in higher impact journals than 
the pesticide toxicity papers, indicating that, 
within the subject area, the evaluations are 
based on science of above-average citation 
impact (p < 0.001 per cent). 


Clinical guidelines: technical 
assessments from the National Institute 
for Clinical Excellence (NICE) 


NICE is a body set up and funded by the 
NHS in England and Wales. (There is a 
separate (and older) body for Scotland, the 
Scottish Intercollegiate Guidelines Network 
(SIGN), formed in 1993 and supported by 
the Scottish Executive Health Department 
which has published 62 guidelines covering a 
wide variety of subjects.) Since the spring of 
2000, NICE has published 47 guidelines to 
clinical practice, of which the large majority 
(33) concern the use of novel drugs. Some of 
these have been contentious and have been 
disputed by the companies involved or by 
patient groups (Lancet, 1999; Kmietowicz, 
2000). The guidelines are accompanied by 
leaflets for patients and depend on 
Technology Assessments (TAs) 
commissioned from specialist units in UK 
universities, notably York, Birmingham and 
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Sheffield. TAS[2] contain an average of 118 
references of which 67 are papers in journals 
covered in the SCI or SSCI, but only 49 of 
these are classified as articles, notes or 
reviews. Those published in 1980 and later 
have been identified, and their bibliographic 
details downloaded and analysed. 

Figure 4 shows their distribution in terms 
of dates of publication, relative to the dates 
of the TAs. It also shows the distribution of 
dates of papers referenced in a sample of 
UK biomedical papers from 2001 
(n =1,142; 35,484 journal citations) and 
those cited in the St George’s guidelines 
(Grant et al., 2000). It is immediately 
apparent that the TA references (NICE) are 
much more recent than both the other sets. 
Their median age is only 3.3 years 
compared with 5.2 years for the SCI 
citations and 7.5 years for the ones from the 
St George’s guidelines. 

Figure 5 shows their nationality, 
compared with the countries’ presence in 
world biomedical literature from 1995- 
2000, when most of the TA references were 
published. As was found earlier (Grant et al., 
2000), the UK is over-represented by a 
factor of 2.2, but several other European 
countries are also over-represented, notably 
Sweden (x 2.2), Belgium, Denmark and 
The Netherlands (x 1.8), Switzerland (x 
1.4) and Austria and Finland (x 1.3). 
Canada is over-cited by x 1.9 and the USA 
by x 1.2. The “losers” are Japan, with only 
11 per cent of the expected references, 
followed by India (13 per cent), Brazil and 
Russia (24 per cent), 

The distributions of these references by the 
research level and potential impact category 
of the journals in which they are published are 
shown in Table III and compared with a large 
sample of biomedical papers in the SCI for 
1997. The distributions are clearly very 
different. As was found earlier (Grant et al., 
2000), the papers are largely clinical, almost 
89 per cent being in the “clinical observation” 
(RL = 1) or “clinical mix” (RL = 2) categories 
compared with under 40 per cent for the SCI 


Table II Comparison of distributions of references on CAC evaluations (n = 1,000) and of pesticide toxicology 
papers in the SCI (n = 5,461) by research level (RL, 1 = applied, 4 = basic) and by potential impact category of 


journals (PIC, 1 = low, 4 = high) 


RL=1 RL=2 RL=3 RL=4 PilC=1 PIC=2 PIC=3 PIC=4 





CAC references 3.2 
Pesticide toxicology papers 1.9 


25.8 
26.8 
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Figure 4 Delays between publication of biomedical papers and their citation on 46 health technology assessments 
for the National Institute of Clinical Excellence (NICE), by UK biomedical papers in the Science Citation Index (SCI) 
and on a sample of 15 clinical guidelines from St George's Hospital analysed by Grant et al. (2000) 
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Figure 5 Relative presence of 20 leading countries in the addresses of papers cited by 46 health technology 
assessments for the National Institute of Clinical Excellence (NICE) and on the world biomedical literature in the 
Science Citation Index (SCI); integrer counts 
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Table III Distributions of references cn NICE technology assessments (n = 1,711) and of a large sample of world 
biomedical papers in the SCI (n = 59,731) by research level (RL, 1 = applied, 4 = basic) and by potential impact 
category of journals (PIC, 1 = low, 4 = high) 

RL=1 RL=2 RL=3 RL=4 PIC=1 PIC=2 PIC=3 PIC=4 
NICE technology assessments 35.8 53.1 9.9 0.5 24.6 29.1 25.2 22.3 
World biomedical papers in SCI 14.7 24.8 27.6 32.9 32.7 30.8 21.0 15.5 
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papers, and there are hardly any papers 
classified as basic research (RL = 4). Despite 
this, the papers cited by the NICE TAs are 
published in journals of high impact, with 
48 per cent in journals of the top two 
categories compared with 36 per cent for the 
SCI papers, the differences being highly 
statistically significant. 


Public perceptions of biomedical 
research advances 


In developed countries, there are many 
potential sources of information for the public 
on health. In recent years, the Web has 
become widely available to the public and has 
been influential. It can be used by people 
suffering from rare diseases, and their 
caregivers, to obtain information and 
emotional support from fellow sufferers. One 
consequence is that patients are sometimes 
better informed than their doctors and may 
press them to provide forms of treatment that 
are newly available. They may indeed take 
matters into their own hands and order 
prescription “ethical” pharmaceuticals without 
the detailed examination, and possibly 
counselling, that should take place. They may 
also purchase genetic tests, sometimes ones 
that are actively promoted by an interested 
party, which may seek to spread anxiety among 
people in the interest of sales[3]. But the Web 
is not all bad: it may enable some patients, 
particularly men, to obtain useful information 
without the embarrassment that could attend a 
visit to their doctor. 

Radio and television have been traditional 
sources of health information for many 
people. In the UK, the radio serial, Mrs Dale’s 
Diary was promoted by the Ministry of Health 
so that useful advice could be disseminated 
almost subliminally through the anodyne 
musings of the wife of a general practitioner. 
Currently, television dramas about hospitals, 
such as ER, Casualty and Holby City, bring 
high-technology treatments to a wide 
audience but may give the impression that 
they save more lives than the less glamorous 
work of public health and preventive 
medicine. Science programmes such as 
Horizon and Equinox can explore issues in 
depth, but the effects may be negated by 
dramas, which give the appearance of veracity 
while actually being based on somewhat 
dubious science. A recent example is the BBC 
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play, Fields of Gold, about genetically modified 

crops. There are also news reports on medical 

research, similar in tone to those appearing in 
newspapers. In fact, a survey by the British 

Broadcasting Corporation (BBC) found that 

science was more popular with viewers than 

sport, foreign affairs, the Royal family and the 
arts (Patel, 1994). All these reports contribute 
to the public perception of biomedical 
research but they are extremely hard to codify 
and quantify. 

Newspapers have the great advantage of 
being a permanent record. Articles in them 
can be treated almost like scientific papers. 
They can be examined at leisure and have 
properties that can be readily codified in a 
database such as: newspaper title, publicaticn 
date, page number, length (column-cm), 
journalist name(s), article title, sub-fields 
covered, and the details of the cited research. 

The results listed below are taken from a 
pilot study covering two months in the 
summer of 2001 which have been given in 
detail elsewhere (Lewison, 2002b). The main 
findings were as follows: 

* In 20 UK national papers, there were a 
total of nearly 200 citations per montt.. 
The Daily Mail and The Daily Telegraph 
provided the strongest coverage. 

+ Most articles were between 30 and 60 
column-cm and had named journalists as 
authors. There were 110 different writers 
indicating that most papers used 
correspondents from several different 
areas (e.g. science, technology, medicine, 
social affairs) for their stories. 

* The subjects most frequently occurring 
(10 per cent or more) were dietetics, 
cancer, neuroscience, epidemiology, 
genetics and obstetrics. Cardiology and 
mental health were seldom cited (only 6 
per cent of citations). 

* UK research was over-cited by a factor of 
about four to one; that from the USA in 
line with its presence in the literature (36 
per cent), and that from continental 
Europe was under-cited. 

<- The general tone was optimistic, 
featuring new drugs and treatments, 
rather than scary. Women and babies 
were preferred as subjects to men and the 
elderly. Experiments involving animals 
were reported neutrally but with an 
implicit sense of approval. 

* Most of the research was reported in 
papers in journals, notably the weeklies 
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such as the BMJ, Lancet, Nature and 
Science. New Scientist was frequently quoted 
although it is mainly a secondary source. 

* The research cited is mainly clinical, 
although some basic research is reported, 
even in the tabloids. 

e Nearly all the papers cited were current, 
although a few review articles also 
mentioned research published during the 
previous few years. 


The importance of such studies is that 
newspaper articles are often influential in 
terms of public opinion and hence in policy 
formation, even if they are misleading or 
wrong. For example, a campaign against 
“Frankenfoods” (sic; genetically modified 
(GM) foods) by some newspapers (Economist, 
1999), notably the Daily Mail, has had a major 
influence on the public in Britain. People have 
reacted with some anger to attempts to carry 
out field trials on crops planted with GM 
seeds, and this has also occurred elsewhere 
(Normile, 2000; Vidal, 2001). In the medical 
field, the current anxiety about the triple 
vaccine against measles, mumps and rubella 
(MMR) has been greatly stimulated by the 
assiduous reporting of papers that purport to 
show a link with autism, or are so described as 
doing (Anderson, 1999). This has led to a low 
take-up of vaccination, and in turn to a 
potentially serious public health problem 
(Laurance, 2002; Spooner, 2002). 

There are several areas of medical research of 
potential concern to the public, which might be 
vulnerable to unsympathetic reporting. The 
first is the use of substantial amounts of public 
money and of the heavy reliance of medical 
research on charitable funds, from both 
collecting charities and foundations, whose tax- 
exempt status cannot be taken for granted. ‘The 
second issue concerns the ethics of some 
research, particularly if it involves stem cells 
from foetuses (Chemistry & Industry, 2001; 
Finkel et al., 2001), the use of organs from 
cadavers (Burton and Wells, 2001), or 
biological samples provided by patients whose 
consent may not have been fully informed 
(Loff, 2000). In each of these, there are vocal 
groups who have opposed the research because 
of the source of the materials. At present, the 
researchers seem to be winning the battle on the 
first and third of these but may be losing on the 
second because of what happened at Alder Hey 
hospital in Liverpool (Research Fortnight, 2002). 

A third issue concerns animal 
experimentation. The violence with which 
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Huntingdon Life Sciences plc, its employees 
and shareholders, have been attacked (Braid, 
2000) and the difficulty of building a 
neuroscience research facility in Cambridge 
that would use primates (Times, 2001) 
indicate that this is a potentially difficult issue. 
It is one that the medical researchers might 
lose if the journalists were to adopt a different 
tone in their reporting of medical research. 

But research has become increasingly 
international, with many papers being co- 
authored internationally, and frequent overseas 
travel by scientists to work in each others’ labs. 
There is also concern that research teams may 
re-locate to countries with a more friendly 
regulatory environment or local public 
(Lehrman, 1995; Firn, 2000; Dver, 2002). So 
we need to monitor how the press reports 
biomedical research in many countries, not just 
to see if the tone is sympathetic, but also to see 
how wide the readership is for news of 
important research developments. This will 
inevitably involve an international project with 
newspapers in many countries being scanned 
and their relevant articles coded and entered to 
a database using a uniform protocol. 
Fortunately the size of the project is not so 
intimidating that it could not be tackled with 
relatively modest resources. It would probably 
be about a hundred times the size of a system 
designed for the UK, and might have to record 
about 250,000 citations per year, far fewer than 
are processed for the SCI. Many of the cited 
papers would be the same, i.e. papers would 
receive citations in the press of several 
countries, so it would be more efficient if the 
look-up were done centrally. The database 
would also be held centrally and the details of 
the citing articles would be entered over the 
Web by recorders using a standard interface 
and a thesaurus of subjects. They would in turn 
be able to access the database in order to carry 
out studies for groups in their own country or 
for academic research purposes. 


Conclusions 


Traditional bibliometrics has yielded 
indicators of the impact of biomedical 
research, but these have been relevant to only a 
small part of the complex pattern of routes by 
which it will influence health. Moreover, they 
tend to give greater credit to basic and 
laboratory studies than to patient treatment 
and other clinical work. Other indicators, 
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based on “downstream” effects of the research, 
seem to indicate that clinical work may have 
rather more influence than basic research, as 
would be expected. However there can be 
much longer periods of time needed for the 
work to have a perceptible effect, eight years or 
so being the median time between publication 
and citation on pollution and some clinical 
guidelines. The current NICE guidelines, 
however, rely on much more recent work. The 
reporting of research in newspapers is 
extremely quick and this dissemination route 
will be of increasing importance in future, 
partly because of this very speed. However 
newspaper coverage will inevitably be selective 
and sometimes misleading or even 
tendentious, which could have unfortunate 
consequences for the research process. 

The other main theme of this paper has been 
that research impacts may well be national 
rather than international, that they will need 
national indicators to capture them and that 
such indicators will need to be developed, 
preferably on a routine basis. The relative over- 
reporting of national biomedical research in the 
UK by its clinical guidelines (Grant et al., 2000 
and the present work) and its newspapers 
(Lewison, 2002b) is probably found in other 
countries. This finding shows the importance 
of a national science base, so that there can be a 
critical judgement brought to bear on science 
carried out abroad and the benefits absorbed 
nationally. The small study on pesticide 
toxicology also suggests that a national science 
base is important to reinforce a country’s voice 
in international discussions on the health 
effects of environmental pollutants. 


Notes 


1 On London buses, the penalty for fare evasion is 
only £5, but for smoking it can be £1,000. 

One TA covers two different guidelines (Nos 3 and 
6) so there are only 46. 

An example is haemachromatosis, with dire 
warnings to 33 million Americans “at particular risk 
for carrying the gene mutations for this potentially 
fatal disease” from the American 
Haemochromatosis Society. 


2 


3 
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Introduction 


The Labour Government has entered a 
commitment to turn Britain into the most 
advanced online nation in the world. This 
undertaking is aimed at creating an 
infrastructure through which private citizens, 
businesses and communities are linked to the 
Internet and are able to use it for a variety of 
personal and commercial transactions. A 
significant part of this development will be the 
provision of online government services. 
Driven forward by the Office of the e-Envoy, 
the Government has announced three 
primary objectives. First, to make the UK 
the best environment in the world for 
e-commerce by 2002. Second, to ensure that 
everyone who wants it has access to the 
Internet by 2005. Third, to make all (or 
virtually all) government services available 
electronically by 2005. 

To achieve its objectives, the Government 
must work to close the so-called “digital 
divide”. This concept identifies a division in 
society between people who have access to 
and the ability to use new information and 
communication technologies (ICTs) and 
those who do not. It has long been recognised 
that new ICTs are not taken up by everyone 
at the same rate (Rogers, 1962, 1995). There 
are those (innovators and early adopters) who 
are more venturesome and are quick to adopt 
any innovations and then there are those (late 
majority) who are much slower to do so. A 
few (the laggards) may never try out new 
ICTs. 

Receptivity to new technologies has been 
associated with demographic factors such as 
age, socio-economic class, level of education 
and disposable wealth (Reagan, 1995; Lin, 
1998; Dupagne, 1999; Wei, 2001). Younger, 
better-off and better-educated people are 
more likely to be early adopters of new ICTs 
(Garramone et al., 1986; Lin, 1998), 

In the UK, the Government has taken steps 
to overcome the digital divide particularly 
with respect to the provision of online public 
services. It has recognised that some social 
groups may lack personal reception access for 
various financial, social or educational 
reasons. Major investment has therefore been 
made in UK online centres in local 
communities around the country. These 
centres — many situated in public libraries — 
will offer free or low-cost access and 
elementary training in the use of computer- 
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mediated communication (CMC) such as 
e-mail and the Internet. These online centres 
will help people develop the skills to use the 
Internet to access information and will teach 
them how to send an e-mail via a personal 
computer, mobile phone or digital interactive 
television (DiTV). By the end of 2002, 6,000 
centres should be in operation (4,300 in 
public libraries). 

In 2001, there were 520 central government 
services to citizens or business, of which 256 
(51 per cent) were electronically enabled. The 
Government’s target is for 513 of these 
services (99 per cent) to be provided online by 
2005. 


Health information online 


Amongst the public services destined for 
online provision is a range of health-related 
services. The policy document Information for 
Health was underpinned by the notion that 
widespread public access to the right kind of 
health information is a core ingredient of 
modern health care (NHS Information 
Authority, 1998). The realisation of this 
online objective for health has been manifest 
in terms of the major NHS Direct initiative 
that deploys a battery of information services 
available over the telephone, on the Web, via 
touch-screen kiosks and DiTV. Mobile 
communications may yet provide a further 
platform for these services. 

Behind all these initiatives lies the 
assumption that the very act of providing 
people with information and access to online 
transactional services (such as booking 
appointments with health professionals, 
online consultations, etc.) will lead to a better 
health outcome for society. In theory, patients 
will be able to consult with their doctor or a 
nurse or other type of health professional over 
remote text, audio or video links from within 
their own homes. It has been hypothesised 
that such a system will offer greater 
convenience for patients and service 
economies for the National Health Service 
(NHS). To what extent, however, will the 
anticipation be matched by the actual success 
of such services? Given the cost of up-front 
investment in ICTs that is required to put 
these online services in place, the effectiveness 
with which they eventually operate cannot be 
left to guesswork and should not depend 
solely on long-term trial-and-error 
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experiences. There is a need for systematic 
and carefully contrived formative research to 
explore how such services will be used and 
how people will respond to different types of 
online application on different 
communications platforms. 

It is already known that patients do 
welcome health information that they can 
take away and peruse at their leisure. 
Furthermore, they attach credibility to the 
information they receive from authority 
figures such as their GP. Written information 
can increase patient compliance with their 
doctor’s instructions (Arthur, 1995; Ley, 
1982). Information leaflets have also been 
found to contribute to better health outcomes 
(Greenfield et al., 1985; Mazzuca, 1982). The 
provision of information online is another 
matter however. Important questions have 
already been raised about the quality of online 
health information, especially where the 
source of that information or its authority and 
credentials are not disclosed (Ferguson, 2002; 
Impicciatore et al., 1997). Yet, there is 
evidence that patients are turning to online 
sources such as the Internet and are using 
health and medical information they find 
there to second-guess their GPs (Rumbelow, 
1999). 


Access and the user 


The integration of technology with the 
provision of public services is regarded by 
Government as an important democratic 
development. It is all about putting the citizen 
first. The prevailing perception is that the 
public wants better access to services and that 
putting them online is one way of satisfying 
the public’s needs. The emphasis so far, 
however, has centred very much on 
technology access. The Government’s priority 
is to ensure that as many people as possible 
are linked to online technologies. In parallel 
with this development, there has been a drive 
to get as many public services online as 
quickly as possible. To facilitate ambitious 
technology roll-out targets, the broad aim is 
to provide online services over as many 
communications technology platforms as 
possible. This means using the PC-based 
Internet, touch-screen kiosks, mobile phones 
and other mobile technologies (PDAs), and 
DiTV (cable, satellite, broadband). While 
access to technology is a prerequisite of online 
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service use, it does not by itself guarantee that 
use will take place. 

The multi-platform scenario is likely to 
deliver greater service reach, since different 
communications technologies are not taken 
up to the same extent by different segments of 
the population. New ICTs can take time to 
become established in a usability context, 
even though access has been widely achieved. 
Upgrades to longer-established technologies 
may prove more fruitful because those 
technologies are already widely used. The 
latter distinction is illustrated by the 
respective potential of the PC-based Internet 
and DiTV. Television as a technology is 
already widely established, while the Internet 
is the new kid on the block. The full potential 
of the Internet as a service delivery technology 
will not be realised until a broadband 
platform has been more widely rolled out at a 
price people are prepared to tolerate. This is 
because broadband technology will upgrade 
the capacity of the Internet to facilitate the 
speedy and effective transmission of complex 
and dynamic multi-media formats that will 
bring this medium into more direct 
competition with broadcast television as a 
source of home entertainment and 
information. 


User psychology 


Even established technologies such as 
television may not guarantee the success of 
online public services — in terms of their use — 
when utility of such services requires a 
different psychological orientation towards 
the medium. This issue goes beyond the 
acquisition of basic skills in ICT use. 
Certainly, while we may be able to access the 
Internet via a desktop PC at home, our ability 
to use it will restrict the effectiveness with 
which it is used. In addition to this fairly 
obvious observation though is a less 
frequently considered issue of whether we feel 
comfortable engaging in any kind of 
transaction over the Net. This last 
consideration is a psychological one. 
Television may be a familiar technology to 
most people and certainly more familiar than 
the PC-Internet. Television, however, is 
traditionally a passive medium. Accessing 
online public services on a DiTV platform 
invokes a shift in psychological orientation, 
whereby television is treated as an interactive 
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medium. Another way of putting it is to say 
that the PC is a “lean forward” technology, 
while the television is a “lean back” 
technology. With a computer, users are 
accustomed to engaging with the technology 
interactively because that is part of the 
inherent nature of the machine. With a 
television set, users are accustomed to 
switching it on and letting its content wash 
over them in a more passive mental mode. 

Hence access does not mean success in the 
context of online public service roll-out. Less 
familiar technologies have to become not only 
more widely distributed but also more 
familiar to more people. With more familiar 
technologies that have established usage 
patterns and psychological orientations, users 
must be re-conditioned to approach them in a 
different way. Television viewers must be able 
to shift readily from watching mode to 
transacting mode. The achievement of this 
objective means that more attention should be 
focused on usability issues. 


Usability issues 


Online public services must be easy and 
effective to use if they are to be accepted and 
utilised as widely as Government would like. 
During the development of such services, 
therefore, more attention should centre on 
establishing the “usability” and “application 
effectiveness” of these services for users. 
Usability here is taken to mean that the 
interface through which users engage with 
online services is easy to understand and 
utilise. Application effectiveness means that 
the most appropriate communications 
formats are utilised for given services. These 
factors underline the need to understand user 
psychology as well as demography. 

In considering user-related issues with 
online technologies, there are four sets of 
assumptions that need to be dispelled. These 
are concerned with the availability of 
technologies, their access, their use, and their 
effectiveness. These issues can be 
encapsulated in four propositions: 

P1. Platform availability does not equal 
platform use. 

Platform sccess does not equal service 
Access. 

Service sccess does not equal 
information access. 


P2. 


P3. 
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P4. Service use does not equal application 
effectiveness. 


Each of these propositions will be considered 
in turn with supporting evidence from 
relevant empirical research. The platform 
principally under consideration here is digital 
interactive television. Most of the evidence 
will derive from research conducted to 
evaluate the provision of online health 
information via television. 


Platform availability + platform use 

The argument under this heading is that 
merely because people have a technology 
platform available to them does not mean that 
they will use it. This point is underscored by 
evidence for the UK on penetration and use 
of PC-Internet and DiTV technologies. 

In 2001, a survey commissioned by the 
Independent Television Commission (ITC) 
found that 35 per cent of television set owners 
in the UK (98 per cent of the population) 
owned PCs with Internet access (Towler, 
2002). The same survey reported that 40 per 
cent of UK television viewers had access to 
DiTV (24 per cent via satellite, 11 per cent via 
cable and 5 per cent via terrestrial 
transmissions). 

Among DiTV viewers, one in five (19 per 
cent) said they used interactive services on 
this platform at least once a week. A clear 
majority of these viewers (62 per cent), 
however, said that they never used interactive 
services. Men (46 per cent) reportedly used 
interactive services more than did women (29 
per cent). Non-use of interactivity on DiTV 
was especially high among the over 65s (70 
per cent). While interactivity was fairly 
limited, amounting to choice of camera angles 
in sports broadcasts and access to more 
detailed story content in news broadcasts, 
what these findings indicate is that availability 
to interactive services on television does not 
guarantee their use. Furthermore, the 
interactive services available at the time of this 
survey were not complicated to use. 

Other evidence has indicated: that viewers 
are prepared to interact with their television 
set if an application really grabs their interest. 
Sky Digital, the direct-to-home satellite 
broadcaster in Britain, released an interactive 
game called Gamestar in March 2002 and 
attracted 1.4 million players a week. These 
games, however, attracted mostly young 
people and the success of DiTV as a purveyor 
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of public services in the future will depend 
upon its ability to persuade a much wider 
demographic cross-section of the public to 
treat television in a “lean-forward” manner. 


Platform access # service access 

If DiTV is to prove an effective platform for 
interactive public services, users must be able 
to find the service they want from among the 
multitude of other services provided in a 
multi-channel television environment. Online 
public services could literally be swamped in a 
digital television package containing perhaps 
200 or more services. Online public services 
in a DiTV environment will therefore have to 
address their “digital visibility” in this 
crowded digital environment. 

Accessing services in this type of television 
environment takes place through the 
electronic programme guide (EPG). This is 
not simply an on-screen television guide that 
provides advance information about 
programmes. It is the gateway to the services 
themselves. The EPG is both a menu of the 
services on offer and an access point through 
to the underlying content — whether 
interactive or non-interactive. EPGs tend to 
be hierarchical, offering at first a short list of 
super-ordinate service or channel category 
headings giving access to a second level that 
offers lists of the services under their specific 
category heading. At the second level, the 
service lists can get quite long, typically 
running to more than three screens. Each 
screen may contain ten services. Some 
channels — usually the historically longest- 
established, mainstream television channels 
and premium subscription channels (e.g. 
movie channels and major sports channels) 
occupy prime positions at the heads of these 
menus. 

The service lists are likely to grow longer in 
the future. Since many interactive public 
services will be offered as “walled gardens”, 
that is services that are separately listed 
alongside mainstream television channels 
(rather than as enhanced services lying 
beneath and linked to sequential programme 
services), they will have to be searched for and 
identified from the service lists. If such 
services are placed far down the list and not 
clearly labelled for what they are, users may 
have difficulty finding them and could easily 
miss them even when they have reached the 
right screen. As evidence to be presented in 
the next part of this paper indicates, television 
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viewers have a low tolerance for working hard 
to find interactive services in multi-channel 
environments. 


Service access $ information access 
The public’s confidence in online public 
services piped through their television sets will 
be enhanced if these interactive services can 
be used readily. Once a public service has 
been located on a television platform, it must 
be easy to find information on it. This does 
not mean simply being able to locate the 
service among a large suite of other channels. 
It also means that once the service has been 
found that the information and transactional 
capabilities it contains are themselves easy to 
access. This point is especially important 
given that many users or potential users of 
interactive public services on a television 
platform may lack the technology literacy of 
early adopters of the Internet. 

The fact that a service has been accessed 
does not mean then that service information 
can also be readily accessed. This point was 
illustrated in the digital health research 
undertaken by a group of researchers 
associated with ciber (Centre for Information 
Behaviour and Evaluation of Research) at 
City University on behalf of the Department 
of Health. This research group evaluated four 
DiTV pilot projects that tested distinctive 
interactive health information and advice 
services on satellite and cable DiTV and 
broadband platforms. The significance of ease 
of movement from the initial platform 
environment interface to specific service 
content was reinforced in an evaluation 
conducted with one of these four pilot digital 
health projects. 

In this case, a service called NHS Digital 
Direct was produced by Communicopia and 
transmitted on the broadband platform 
operated by Kingston Interactive Television 
(KIT) in Hull. The service comprised text 
and video-on-demand information covering a 
wide range of health topics. The service was 
provided alongside a range of other 
entertainment and information services in the 
KIT environment. Over a pilot period 
extending several months, a number of 
changes occurred to the position of NHS 
Digital Direct on the KIT platform (see 
Nicholas ez al., 2002). 

In an initial period spanning around 12 
weeks, two menu levels were in operation. 
One of these was the platform Home page for 
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the overall range of services provided by KIT. 
Within this was a further Local Link menu. At 
this time, NHS Digital Direct could be 
accessed both directly from the Home page 
and also via the Local Link menu. Users 
therefore had more than one route through to 
the service and did not have to explore very 
far before accessing it. 

In a second period, that spanned around 
four weeks, there were again two relevant 
menu levels: the Home page and the Local 
Link menu. At this time, however, the direct 
link to NHS Digital Direct from the Home 
page was removed. Users now had to click 
through to the Local Link page and then click 
again to reach the health information service. 
In a third period, the menu structure shifted 
again from a two-level system to a three-level 
menu. This comprised a Home page menu at 
level one, a further platform operator menu at 
level two, and then a further page within the 
level two menu that revealed a level three 
menu through which digital health content 
could be accessed. At this stage then, users 
had to navigate three screens to find the 
digital health service as compared with two 
screens in the two earlier phases. 

Digital logs were used to monitor real-time 
use of this service. These logs registered levels 
of use for different parts of the service and 
yielded a number of user-related metrics. 
Measures were taken of the numbers of text 
screens and videos viewed across the three 
periods outlined above. Volumes of use fell 
across these periods as access to the digital 
health service required progressively more 
effort on the part of users. 

From period one to period two, when 
access to the digital health service shifted 
from being one click away to two clicks awey, 
there was a 33 per cent decline in number of 
text screens viewed and a 29 per cent decline 
in numbers of videos viewed. From period 
two to period three, when access shifted from 
being two clicks away to three clicks away, 
there was a 45 per cent decline in text screans 
viewed and a 35 per cent decline in videos 
viewed (see Figure 1). NHS Digital Direct 
attracted an average of 50 users per day who 
viewed about 1,700 text pages and 140 viceos 
during the latter part of period two. This level 
of use fell to 15 daily users viewing about 500 
text pages and 40 videos during period three. 

These findings provide a clear illustration 
that in a “lean-back” television environment, 
interactive public services must be readily 
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Figure 1 Daily average (median) number of (a) text screens and (b) videos viewed by menu period 
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accessible once the service as a whole has 
been located on the platform menu. Any extra 
effort that is imposed upon users by the 
service supplier or platform operator — even as 
little as one additional click on the remote 
control — could have significant implications 
for overall levels of service use. 


Service use + application effectiveness 
Application effectiveness depends upon 
finding the most suitable format for 
conducting certain types of information 
enquiry or communication transaction. 
Traditionally, the choices of communication 
channel have been limited, comprising spoken 
face-to-face, telephone, and written (hard 
copy) forms. In the digital communications 
environment, the options have expanded to 
include video links, electronic mail and voice 
mail. Electronic communications can be 
conducted one-on-one or in a group format. 
Such communications can be asynchronous 
(a message is sent and a reply despatched 
some time later) or synchronous (two-way 
simultaneous communications). Once a user 
has located an interactive service on a multi- 
channel platform and navigated their way 
through to the required service content or 
transactional link, does the format offered 
provide the most appropriate or satisfactory 
modality through which to complete an 
enquiry or transaction? 

Application effectiveness is likely to be a key 
factor that underpins the longer-term success 
of interactive public services. In the health 
context, for example, most users may be 





perfectly happy to search text-based 
information in undertaking a general 
information enquiry. A person suffering from 
a specific medical condition, however, may 
draw additional comfort from viewing video 
that features real people who also suffer from 
that same complaint. One reason for this may 
be that the sufferer may seek emotional 
support and reassurance as well as factual 
information with regard to their condition, 
and this can be supplied more effectively in a 
video format than in a text format. In both 
these cases, the communication links are 
asynchronous. There may be occasions when 
synchronous links are needed. 

Someone suffering from a medical 
condition that is currently giving them cause 
for concern may need to speak to a health 
professional immediately. Watching a pre- 
recorded video will not suffice. They need 
immediate advice or diagnosis. In this case, 
the opportunity to speak to a qualified 
professional on the telephone or actually 
being able to see that person via a live video 
link may be more appropriate. 

One important concept associated with 
application effectiveness in the interactive 
communications context is “social presence”. 
The social presence model claims that 
communication media vary in the degree to 
which they can convey the physical presence 
of communicators (Short et al., 1976). The 
model arranges media along a continuum 
from low (written numeric or textual content) 
to high social presence (e.g. face-to-face 
communication). It is further contended that 
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individuals will select the medium that they 
perceive to have the highest social presence. 

Another related model is media richness 
theory (Daft and Lengel, 1984). This theory 
posits that people select communication 
technologies based largely on the attributes of 
the medium. Again, media can be arranged 
on a continuum from “lean” to “rich” based 
on their speed of feedback, variety of 
channels, friendliness of source and richness 
of language used. Media richness theory also 
considers the context of use and proposes that 
individuals will seek to match the richness of a 
communication medium with the complexity 
of the task for which it is used. 

Some of the earliest research reported that 
newer media were rated by users as more 
appropriate for information exchange tasks 
requiring lower social presence (Trevino et al., 
1987; Rice, 1993). Compared with direct 
face-to-face (FtF) communication, for 
example, computer-mediated communication 
(CMC) was reportedly ranked low in social 
presence (Perse and Courtright, 1993). Later 
research challenged this notion of CMC, 
however. Indeed, engagement with 
computers was reported frequently to take on 
the same tenor as interpersonal 
communication (Reeves and Nass, 1996). 
CMC can be used for socio-emotional tasks 
just as readily and effectively as can FtF 
communication (Joinson, 2001; Parks and 
Floyd, 1996; Walther, 1996). The rules 
determining the nature of communication via 
computers often resemble those that govern 
the way individuals interact face-to-face (Lee 
and Nass, 2002). While certain 
communication cues in the FtF context are 
missing in the CMC context, skilled 
exponents of CMC find ways of enhancing 
the form of communication in that 
environment and may nevertheless engage in 
emotionally fruitful encounters with others in 
that context (Tidwell and Walther, 2002). 

This background is important to the 
establishment of public services online, 
especially where such services may involve a 
range of different types of CMC interaction 
between members of the public and public 
service providers. In the digital health 
research, although not systematically 
investigated, the significance of social 
presence and media richness emerged 
through a number of the findings (see 
Nicholas et al., 2002). 
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The digital health television pilots provided 
information in text and video form and 
offered access to transactional services such as 
online appointments booking with GPs and 
other health professionals and live telephone 
and video links to NHS nurses. Text 
information about medical conditions that 
was provided online via the television was 
valued by many of its users. 

In relation to one pilot operator that 
provided a largely text-based information 
service via cable television in Birmingham, a 
postal survey of self-identified service users 
indicated that over four in ten (44 per cent) 
sought information relating to their own 
medical condition, while one in four (25 per 
cent) sought it for friends and family. Over 
eight in ten users of this information (81 per 
cent) said they found the text information 
useful. Over four in ten users (43 per cent) 
said that the health text information helped a 
lot in dealing with their doctor. 

There were other individuals, however, who 
did not use this digital health service even 
though it was available to them at no extra 
charge as existing cable subscribers. Among 
these non-users, lack of use among some was 
attributed to not being very good with 
interactive technology (17 per cent) or not 
being able to find the content they wanted 
within the digital health service (7 per cent). 
Far more of these people, however, said that 
their doctor tells them all they need to know 
about their medical condition (24 per cent) or 
that they preferred to receive printed 
information from their doctor about a 
condition (40 per cent). These findings 
suggest that avoidance of an interactive 
television health information service may 
occur among some potential users because 
they prefer to go direct to their doctor for 
such information. DiTV may lack sufficient 
media richness for these individuals as 
compared with direct physical contact with 
the GP. 

The same digital health television service 
provided access to a transactional link with a 
local NHS Direct call-centre. Through this 
service, users could telephone the call-centre 
and simultaneously see on their television 
screen the nurse to whom they were talking. 
The user at home could ask the nurse any 
questions about their medical condition and 
the nurse would offer advice and information. 
The visual element could be further utilised 
by the nurse who could show pictures, charts 
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and video clips to the user where they proved 
relevant as part of the response to the enquiry. 

Users on this online video link found the 
service helpful. What was particularly 
interesting, however, was that many callers 
(36 per cent) sought reassurance as much as 
specific medical help. Users offered 
comments such as “you feel reassured by her 
[the nurse’s] body language” and that it is 
“easy when you can see her face ... a lot more 
personal ... more human”. The encounter 
was described as “like being face-to-face ... 
felt as though she was there”. One caller 
observed “when you talk blindly on the phone 
you have no idea whether they are just going 
through the motions” (Nicholas et al., 2002). 
Again, these remarks provide qualitative 
evidence that is consistent with the notions of 
media richness and social presence. In this 
example, video links offer a richer format than 
text or even audio links. For certain types of 
health-related enquiry, users need to see as 
well as hear whom they are dealing with. 
Where there is an emotional need in addition 
to an information need, pictures may speak 
louder than words. 


Conclusions 


The roll-out of digital technology is obviously 
essential if society as a whole is to reap the 
benefits of online public services. Without 
access to the delivery technology, the public 
cannot receive electronic services. Putting the 
infrastructure in place has therefore been at 
the forefront of planning for the future digital 
world. As the analysis presented in this paper 
has indicated though, infrastructure is not 
everything. Even when virtually all public 
services have been e-enabled and most people 
have reception technology either at home or 
within easy reach at a public location in their 
community, the effective utilisation of these 
services requires a number of other important 
considerations. 

Two key elements that will underpin the 
success of online public service provision are 
the usability and application effectiveness of 
these electronically communicated services. 
These two concepts invoke a change of 
orientation towards understanding digital 
consumer psychology. In particular, there is a 
need to recognise that even when adapting an 
established medium such as television to 
provide interactive capability, viewers will not 
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universally shift their mindset towards the 
medium. While there may be certain 
advantages in utilising television in the 
context of closing the digital divide, where the 
latter is envisaged primarily as a matter of 
demographics, such a development omits to 
take fully into account the significance of 
audience psychology. 

Even when most viewers have been 
cultivated to regard television as a “lean 
forward” as well as a “lean back” medium and 
can engage with the medium in both ways, 
there remains the issue of finding the best 
formats for particular kinds of information 
enquiry and for conveying specific types of 
transaction. Recognition of the differential 
social presence and media richness afforded 
by different communication formats is a 
matter of paramount importance. Future 
research with CMC in the public service 
context must establish the most appropriate 
and effective formats for different types of 
online task. Only then can the digital divide 
truly be closed and online public services 
enjoy longer-term acceptance and success. 
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Abstract 


NHS Direct Digital was available to almost exactly the 
same number of users in both Hull and London via two 
different transmission DiTV companies: Kingston 
Interactive Television (KIT) and HomeChoice. This paper 
employs metrics based on server generated logs to 
compare use and user differences between the two 
services. Server logs that record user activity on a real- 
time and continuous basis were made available and used 
to make comparisons between the two services. The data 
were collected for both services over a similar period, 
February 2002 to May 2002. During this period the NHS 
Direct Digital service was visited by approximately 3,000 
people and more than 5,000 NHS Direct videos were 
viewed. Making the study probably the largest analysis of 
health videos ever conducted. 
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Introduction 


There is a considerable literature on Web 
metrics, but very little, if any, on digital 
interactive television (DiTV) metrics. This is 
not surprising because of the relative novelty 
of DiTV, but given the rapid growth and 
importance of this medium metrics will soon 
have to be developed and pressed into service 
to chart progress and effectiveness of this 
much hyped medium. In fact DiTV pages are 
stored and delivered to the user in much the 
same way as Web pages. Hence much of the 
research conducted on Web metrics is 
applicable to the DiTV environment. Web 
metrics have been developed over a relatively 
short period of time (Stout, 1997; Williams, 
1999; Gutzman, 1999; Nicholas et al., 1999). 
Indeed, most of the metrics are not wholly 
new and their origins can be traced back to 
research on OPAC log files (Peters, 1989; 
Nicholas, 1996). Typically, the metrics used 
are the number of pages or screens viewed 
(page impressions), number of users/visitors, 
time spent viewing a page, number of pages 
viewed per search session, session time length 
and the number of screens printed or 
downloaded (Nicholas et al., 1999, 2002). 
Time measures are calculated from the date 
and time stamp field and pages viewed by 
counting and verifying transaction lines. 

DiTV logs do follow the Internet 
conventions of recording the time that the 
client requests the page, although a log off time 
is not recorded and, as a consequence, this 
introduces problems with dealing with highly 
skewed data. There are, however, important 
differences between the two platforms that 
have consequences for the metrics employed. 
Thus, unlike the Internet, DITV pages are not 
normally cached to a client’s machine, 
although DiTV pages may be cached if the 
service is routed through local distribution 
server hubs. Furthermore, there are no spiders 
and robots actively collecting information in 
the DiTV network that have to be discounted. 
The DiTV network is much more like an 
intranet where users are paid up members 
accessing a server. And thus it is possible to 
identify individual users, something almost 
impossible to do on the Web — and this is a 
huge bonus for researchers. 

As part of an ongoing Department of 
Health funded research project[1] evaluating 
the impact of DiTV health channels the 
authors have been testing metrics to chart the 
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use and impact of these channels. This work 
is reported here. 


Aims, objectives and scope of the study 


The aims of the research were to use server log 
based metrics to compare and contrast two 
DiTV information transmission services 
delivering the same content, but to two 
different communities in the UK. The content 
in this case was, unusually, 110 NHS videos 
covering such topics as diabetes, blood 
donating and childhood eczema. All the videos 
were produced by Communicopia Ltd which 
was also responsible for negotiating with the 
distribution companies regarding transmission. 
In addition to the ubiquitous “visitors” 


(individual identifiable addresses) and pages ` 


viewed metrics, the following were also 


employed: 
* average number of videos viewed per 
visitor; 


e session types (single page sessions, single 
daily session and multiple user sessions in 
a day) and session numbers; 

* reach (the percentage of those who had 
access to the service who viewed the 
service at least once during a fixed time 
period); and 

e  re-visits to the site. 


Site backgrounds 


Two services were compared: that of NHS 
Direct Digital distributed by HomeChoice, 
which has about 10,900 subscribers; and 
NHS Direct Digital distributed by KIT 
Interactive, which has 10,000 subscribers. 
The current paper looks only at the logs of the 
on-screen information services of each. 
However, it is worth describing all of their 
services by means of providing context. 


NHS Direct Digital via Home Choice 

The “NHS Direct” via HomeChoice service 

consists of two elements: 

(1) limited text and images; and 

(2) “on-demand” short video programmes 
(the logs of which are analysed in this 


paper). 
The HomeChoice was available to cable 


TV-on-demand subscribers in the London 
area. The service was predominately a video 
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service. The main menu of the service is 
structured on one main menu before arriving 
at a screen from which the user can download 
a video. 


Text service 

Text is displayed by overlaying the text on to a 
video background. The main text elements, 
other than the opening video/text 
introduction and main menu, were details of 
local hospitals. The hospital details were text 
and graphics overlaid onto a moving video 
background. The only other use of text was 
for the quizzes and questionnaires, and these 
again were overlaid onto a moving video 
background. 


Video service 

This consists of 110 videos. However, videos 
are grouped into topics. A topic such as 
coronary heart disease will be covered by a 
number of short videos accessible 
independently and in the order decided by the 
viewer. They are thus, as far as possible, self- 
contained. The service is structured so that a 
video can be downloaded two screens from 
arriving at the service. 


“NHS Direct” via KIT 

The “NHS Direct” via KIT service consists 

of: 

* text, images, and audio; 

e “on-demand” short video programmes, 
the logs of which are analysed in this 
paper; and 

* an online immunisation diary, whereby 
subscribers could maintain personal 
immunisation records on their TV set. 


The NHS Direct service, via KIT, was 
available to broadband digital TV-on-demand 
subscribers in the Hull area. The main menu 
of the NHS Direct service is structured on up 
to four hierarchical menus before arriving at 
information content. The content pages list, 
on the right hand of the screen, the other 
menu options at the same hierarchical level 
for that particular topic. The principles of 
navigation are that numbers are used to move 
forward and backwards through pages and 
menus, rather than the words “next page” or 
“previous menu”. The content of this NHS 
Direct service differs from the HomeChoice 
in that it has much more text information. 


Text service 
This is distributed over a number of menus, 
the main ones being: 
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* Not feeling well? This part of the service 
allows the viewer to choose a part of the 
body (head and chest, abdomen, limbs, 
or skin) and to choose from a list of 
possible symptoms, or to choose from a 
full list of all 54 symptoms. 
(Unfortunately, this list has not been 
sorted alphabetically, but puts the four 
previous menus on top of each other!) 
The viewer then has to answer yes or no 
to a series of questions which leads them 
to a possible diagnosis and a suggested 
course of action (e.g. see GP, phone 999). 

• A-Z of conditions. This section lists 274 
conditions. The section contains 3,661 
pages of information. For each condition, 
information includes symptoms, causes, 
diagnosis, prevention and treatment. 

There are also sections on “First aid”, 
“Medicine cabinet” (a searchable A-Z 
index of 149 medicines), “Healthy living” 
and “Local information”. The latter 
includes information on blood donation 
(including a video) and details of local 
doctors, hospitals and pharmacies. 


Video service 
The “on-demand” video collection comprises 
12 introductory menu videos and 28 video 
content or topics. The former give an overview 
of the service and its navigation and features. 
The latter consists of approximately 100 
videos. A topic such as coronary heart disease 
will be covered by a number of short videos 
accessible independently and in the order 
decided by the viewer. They are thus, as far as 
possible, self-contained. The videos are made 
available as sub-menus to the text service. 
The video on-demand service is an 
innovative and new way of presenting health 
information and issues related to usability of 
the service were considered a concern. In focus 
groups[2] with non-DiTV users, participants 
struggled to select and play multiple videos. 
Further, some participants did not take in the 
instructional text and so did not realise the 
need to tick videos. Others read the text, but 
having ticked the videos were unsure how to 
play them. They were looking for an obvious 
route to play the video, for example, the word 
“play” or a “play video” icon. This finding was 
at odds with data collected by questionnaire of 
users of the NHS Direct text and video service 
on KIT, only 6 per cent of users said it had 
been difficult to access the health video service. 
KIT is a video on-demand service and NHS 
videos were introduced without regard to how 
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users normally download videos. While quirks 
of the system may seem unintuitive to focus 
groups (non-users) it is thought that usability 
was not such an issue with the larger KIT or 
HomeChoice subscriber base. Questionnaire 
evidence did point to difficulty in using the text 
and video parts of the KIT service by older 
users. Users aged 55 and under generally 
found KIT easier to use compared to older 
users. Female older users in particular found 
the service difficult, though females under 36 
found it easier and this group seemed 
confident in using the technology. 


Methodology 


Log files are machine-generated records of 


- user activity. DiTV logs are similar to those of 


other digital platforms and a detailed 
explanation of how the data is analysed can be 
found in a number of articles published by the 
authors (Nicholas et al., 2001a, b). 

Log files related to videos are a record of 
video requests delivered by the server to the 
client. Typically the information recorded is 
the identification of the client, the time and 
date of each video delivered and the name of 
the video delivered. The exact amount of 
information collected will depend on the 
software used and how the server was 
configured. A typical DiTV video log line 
(from NHS Direct Digital) locks like: 

Feb 1 00:23:06.270 924370037 

nhs_00aq00039.mpi 

Feb 1 00:22:12.870 924370037 

nhs_00aq00100.mpi 

Feb 1 00:26:03.970 924370037 

nhs_00aq00096.mpi 

Feb 1 00:30:54.010 924370037 

nhs_00aq00047.mpi 


The lines in the script give the following data: 

e Feb 1 00:23:06.270 records the date and 
time of the request. 

* nhs_00aq00039.mpi is the identification 
number of the video downloaded and is 
looked up via a database procedure to 
extract the video title name. 

e 924370037 is the user’s ID number. 


Across the two DITV services examined here, 
logs were available from February to May 
2002. During this period NHS via 
HomeChoice registered 1,581 individual 
visitors and via KIT 1,594 visitors — very, very 
similar numbers. 
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Data were obtained from an analysis of 
server log files, only server logs related to 
videos were compared: 

(1) NHS Direct service via KIT. Logs of this 
service were recorded on two servers. 
Transactions related to the view of text 
pages and text menu screens were 
recorded on a server held by 
Communicopia. Views to videos were 
recorded separately by KIT. The two sets 
of log files were made available to the 
research team however in such a way that 
two files of use could not be combined. 
The log text file did not record the user’s 
unique identity number instead users were 
allocated a floating session identity 
number. A unique user number was 
recorded on the video transaction log files. 

(2) NHS Direct service via HomeChoice. Log 
files were recorded and stored in a 
database. Fields related to individual 
transactions were made available to the 
research team. These fields included a 
field recording the user’s unique 
identification number, as well as screen or 
video view and time of download. 


Respondents’ characteristics 

A questionnaire sent to subscribers of each 
service indicated that HomeChoice had more 
female users for its NHS Digital service (52 
per cent) — see Figure 1 for details. The 
reverse was true for KIT, more men (55 per 
cent) were found to be users. But for both 
transmission services most users of NHS 
Direct fell within the age group 35 to 54 
(Figure 2). However, users via KIT tended to 


‘igure 1 Gender distribution of NHS Direct Digital users (KIT and 
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Figure 2 Age distribution of NHS Direct Digital users (KT and 
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be older: 41 per cent of users via KIT were 
over 55 years old, whilst only 7 per cent of 
HomeChoice users fell into this age bracket. 
Those connecting to the NHS service via 
HomeChoice tended to be younger: 47 per 
cent were aged under 35 compared to 21 per 
cent of KIT users who fell into this age grous. 


Results 


User numbers 
The services were compared over the four 
month period February to May 2002. During 
this period a total of 3,175 individual users of 
the services were identified. A reasonable take 
up from a standing start — there had never 
been any service like it as far as these users 
were concerned, The NHS Direct service via 
KIT was used by 1,581 users, whilst 1,594 
HomeChoice users had used the service. The 
reach figure, the percentage of subscribers 
who had used the service, for both was about 
16 per cent. The data are amazingly similer 
given the very different nature of the two 
communities (Hull is not London after alb. 
Figure 3 gives the daily plot of NHS Direct 
Digital users broken down by delivery 
company. The plots between HomeChoic2 
and KIT are, apart from the peaks, quite 
similar. The HomeChoice service did starz at 
a higher level, approximately 84 daily users, 
compared to about 51, and relates to a peak 
enjoyed by the service at inception (25 
January). By mid February the gap between 
services narrowed and both were recording 
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Figure 3 Daily plot of NHS Direct Digital users broken down by delivery 
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numbers within a band of 45 to 55 daily users. 
There was a decline in user numbers over the 
period. The number of users by the end of the 
survey period was just ten to 20 users per day. 
A regression over the period showed that the 
KIT service was losing about one-fourth of a 
user per day, while HomeChoice was losing 
on average about one-fifth of a user a day. 
However, the decline stopped towards the 
end of the survey period and this suggests that 
long term use would stabilise at about ten to 
17 daily users for HomeChoice and about 14 
to 18 daily users for KIT. 

There are some “outside” factors that 
explain some of the figures. Thus there was a 
loss of service to KIT subscribers between the 
25 February to the 5 March. Further, there 
was a peak of user activity viewing the service 
via KIT on the 8, 11 and 12 April resulting 
from a mail out of a questionnaire that was 
sent to all KIT subscribers. The mail out 
included a letter reminding subscribers how 
to use and find the service — they were plainly 
reminded, and this shows how important 
publicity is for innovative and unusual 
services like NHS Direct. 

Figures 4 and 5 look at user activity by day 
of week over each week of operation. Use on 
Saturdays and Sundays regularly made up 
between 25-30 per cent of user activity. 


The repeat user 

An alternative way of looking at user activity — 
and perhaps a more powerful way — is to 
examine whether the service attracts return 
trade. Coming back to a service constitutes 
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Figure 4 User activity over day of week by week — HomeChoice 
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Figure 5 User activity over day of week by week — KIT 
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conscious and directed use — as good an 
approximation of this as is possible from the 
logs. People might arrive at an Internet site or 
TV channel by accident — and, of course, that 
constitutes “use” according to the logs, but 
they are unlikely to arrive at the same service 
again by accident. Table I looks at the 
frequency with which people return to each 
service over the four months. HomeChoice 
had 418 repeat users and KIT a total of 523. 
KIT had about 25 per cent more repeat 
usage. This is significant and argues that 
HomeChoice had the greater number of users 
who visited once and did not return. 

Those who returned did so on average once 
every six to seven weeks. The average number 
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Table | User visits and return visits within period 








HomeChoice KIT 
si d n % n % 
Once 1,176 73.8 1,058 66.9 
2to5 385 24.2 451 28.5 
6 to 15 31 1.9 64 4.0 
Over 15 2 0.1 8 0.5 
1,594 1,581 
of days between visits of those returning to the 
HomeChoice service was 41 days and 43 days 
for KIT returnees. 
=} Figures 6 and 7 provide returnee 


information for each month and classify users 


as either new users or users returning from 
previous months. The figures compare new 


and returnee users from month to month only 


Figure 6 New users and returnees between months — HomeChoice users 
(absolute figures) 
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y Figure 7 New users and returnees between months — KIT users 
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— return visits within a month are not 
included. 

New users attracted to NHS Direct via 
HomeChoice fell over the period. The 
number of new users fell by 38 per cent from 
March to May (Figure 6). This was not true 
of return visitors. The number of return 
visitors increased month on month and 
increased by approximately 16 per cent 
between March and May. 

NHS Direct via KIT (Figure 7) attracted a 
dramatic increase in new users in April 
compared to March: a little under three times 
as many — 626 compared to 219. This resulted 
from the mail out of a questionnaire to all KIT 
users in April that included a letter promoting 
access to the service and, significantly, how to 
access the service. It had an impact in 
attracting new users to the service. However, 
the figure of new users recorded in May is poor 
compared to both the April figure (ten times 
fewer) and March (three times fewer) new user 
figures. Return users also increased in April as 
a result of the mail out compared to March, 
but not by as much as the increase recorded for 
new users ~ about a 68 per cent increase in 
return visitors. Return visitors in May, though 
higher than the March figure by half as much 
again, were down on the April figure. Clearly 
the mail out had an impact on site use, though 
the benefits of the promotion in attracting new 
users and importantly greater return usage 
may well be short lived. 


Use 

While activity in terms of numbers of users 
was much the same for the two services, the 
same cannot be said about use. Table II 
compares both use and user numbers. The 
amount of use made of NHS Direct Digital 
videos delivered via KIT is about half that for 
HomeChoice. So on this count HomeChoice 
scores most highly. 

Figures 8 and 9 provide a visual of the daily 
and weekly distribution of use. As can be seen, 
use of the NHS service via HomeChoice 
exceeds that of the service delivered by KIT for 
all weeks, except for the week when the 
questionnaire was sent to all KIT users. 
Furthermore, the decline in use in both services 
for February, March and May is surprisingly 
similar, though there is not a sufficiently long 
series to say that the decline is the same. 

Figures 10 and 11 look at all transaction use 
by hour of day and day of week. It is of 
interest to note that the KIT service does not 
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Table II Use and user activity for NHS Direct Digitel 














HomeChoice KIT 
Use Users Use Users 
Month Week n n n n 
February 5 1786 262 529 181 
6 1,487 266 961 330 
7 1,016 231 794 286 
8 1,539 301 397 144 
9 875 160 N/A N/A 
March 10 838 130 551 176 
11 485 105 346 121 
12 576 96 290 102 
13 361 82 292 107 
April 14 511 102 597 186 
15 506 96 2,088 605 
16 492 103 634 198 
17 607 121 401 140 
May 18 666 135 336 114 
19 480 97 344 114 
20 546 95 233 88 
21 389 51 156 64 
Figure 8 Daily plot of NHS Direct Digital uses broken down by 
host service 
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increase in use through the day in the same 
way as for HomeChoice. This might be an 
indication of capacity problems with the KIT 
service, although we do not have any evidence 
to support this claim. The increase in use of 
the service via HomeChoice during the late 
evening and early morning hours reflects the 
fact that HomeChoice has a younger 
subscriber list compared to KIT. 


Looking behind the numbers 
Both services made available the same 
selection of NHS videos. However, the 
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Figure 9 Weekly plot of NHS Direct Digital uses broken down by 
host services 
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Figure 10 Use by hour of day 
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services were very different in terms of menu 
structure and the number of information text 
pages made available. Though both services 
offered text pages the number of text pages 
offered by KIT was considerably more 
extensive. KIT offered approximately 9,000 
text pages covering a wide range of medical 
conditions. Furthermore, KIT used a menu 
structure that incorporated videos that the 
user had to view before they could interact 
with the service. Users had to view an 
introductory menu video the first time they 
engaged with the system in a day, anda 
further video on entering each of the six 
sections of the service, These videos were not 
viewed if the user re-entered the service on 
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Figure 11 Use by day of week 
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the same day. Table III gives a breakdown 
between text, video and menu components of 
each service for the four-month period 
February to May. 

Approximately 27 per cent of the use of the 
KIT service was accounted for by text pages; 
however, this was true for only 9 per cent of 
use of NHS Direct via HomeChoice and 
reflects the reduced text service of 
HomeChoice. Most use, over two-thirds of use 
on KIT and over half of use on HomeChoice, 
relates to accesses to menu objects (including 
introductory videos). In terms of NHS videos, 
37 per cent of use on the HomeChoice and 0.5 
per cent use on the KIT service related to the 
download of information videos. The less than 
1 per cent use related to videos on KIT surely 
represents a failure of this part of the service. 
Plainly NHS videos on KIT have not attracted 
much use. 

Figure 12 plots the weekly use of NHS 
videos on KIT and on HomeChoice over the 
four-month period February to May. There 
has been little or poor use made of NHS 
videos on the KIT service compared to use of 
the same selection of videos on HomeChoice. 


Table IlIi Breakdown of menu, text and video use of 
NHS Direct Digital service 


HomeChoice KIT 
n n 
Menu object (including 7,131 84,070 
introductory videos) 
Video view 4,857 511 
Text pages 1,172 30,953 
Total 13,160 115,534 
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Figure 12 Weekly use of NHS videos on KIT and HomeChoice 
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The number of videos viewed on 
HomeChoice was about 200 videos per week, 
however only about ten videos per week were 
downloaded via the Kit service. A dramatic 
difference by any account. 


Topic videos 

Table IV lists the videos viewed by topic for 
each service. The most popular topic videos 
watched for both services were “Diabetes” 
followed by “Coronary Heart Disease”. 
However, for each topic HomeChoice users 
downloaded about seven times more videos 
compared to KIT users. For example, for 
diabetes videos HomeChoice users 
downloaded 686 videos over the period, while 
KIT users downloaded only 99 diabetes 
related videos. For no video topic were KIT 
users in the majority. 

In general there is quite a variance in the 
take up of videos. However, we would not 
expect the take up of the video services to be 
the same across health topic or across type of 
user in focus groups[3] it was found that 
respondents thought that some material 
would be better presented in text form. This 
was principally in regard with the sections in 
the pain management and coronary heart 
disease programmes. In both cases there was a 
strong feeling that the information on 
medication would have been better presented 
in text, as a list or, more popularly, in tabular 
form. This view may represent an indication 
of the type of information people would prefer 
in text form. It seems to indicate that this is 
where the information is a series of simple 
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Table IV Use by video topic — February to May 2002 





HomeChoice KIT Total 
Video topic n n n 
Acupuncture 47 3 50 
Bypass surgery 66 1 7 
Cataract 267 25 292 
Cervical screening 326 30 356 
Childhood eczema 78 8 86 
Coronary heart disease 392 57 449 
Diabetes 686 99 785 
Facet joint injections 47 7 54 
Healthy eating 328 14 342 
Resuscitating 335 16 351 
Stress incontinence 199 8 207 
Stress and relaxation 143 143 
Ulcerative colitis 120 120 
Urge incontinence 149 21 170 
Choking 127 127 
Foray for health 224 77 301 
Donating blood 255 15 270 
Cryoanalgesia 35 1 36 
Epidural 42 1 43 
Medication 59 7 66 
MMR 69 38 107 
MMR 69 38 107 
Tens 56 56 
Tens 56 56 
The clinic 41 12 53 
The clinic 41 12 53 
What is pain 91 9 100 
What is pain 91 9 100 
Hypertension N/A 35 35 
Hypertension N/A 35 35 
The road to health — Brighton N/A 13 13 
The road to health — Brighton N/A 13 13 





facts (i.e. for taking medication, how it helps, 
and what the side effects are), for consultative 
purposes (unlike, say, patient experiences), 
and information requiring repeat access. 

Note for many of the topics there was a series 
of videos available. For example, there were 
nine videos available under diabetes and eight 
videos available under coronary heart disease. 
For topics where only one video on the topic 
was available, the MMR video (downloaded 69 
times by HomeChoice users and 38 times by 
KIT users) proved the most popular. 

Figure 13 gives the percentage share of use 
by video topic broken down by service. KIT in 
relative terms, though not in absolute 
numbers, was successful in attracting views to 
videos on MMR, diabetes and coronary heart 
disease; while HomeChoice was successful in 
attracting use of views to blood donating, 
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Figure 13 Percentage share of use by video topic — February to 
May 2002 
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healthy eating and resuscitating. The main 
menu of HomeChoice included blood 
donating as a separate item and prominent 
digital visibility of this service on the main 
menu is believed to have encouraged a greater 
use of this video. The video was viewed 17 
times by HomeChoice subscribers for each 
single view by a KIT user. Compare this to 
MMR which was relatively well used by KIT 
users, there was only about two views to this 
video by HomeChoice users for every one view 
by KIT users. The MMR video was the only 
video available under the KIT vaccination 
menu and this may have impacted on use. 

Table V ranks video titles (angrouped by 
topic) by usage. The top ranked 20 videos 
accounted for 47 per cent for all HomeChoice 
videos used and 67 per cent of KIT videos 
used. The reduced percentage recorded for 
HomeChoice suggests a wider use of video 
titles for this service. Videos that are 
highlighted bold are unique to the top 20 of 
that service and do not appear in the top 20 of 
the other service. 

Comparing the unique videos of each 
service it seems that many on the 
HomeChoice service are videos giving 
background information, for example on what 
are cataracts, what is stress incontinence, 
what is urge incontinence, and what is 
coronary heart disease? These videos do not 
appear in the KIT top 20 and perhaps users 
are using the text service on KIT to get an 
introduction to the service and hence feel that 
they do not need to repeat this information by 
seeing the video. HomeChoice users do not 
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Table V Ranked top 20 videos viewed on each service 





HomeChoice 

Top 20 video viewed (ranked) % 

Healthy eating 6.8 
Blood donating 5.3 
Resuscitating an adult 3.7 
What is diabetes? 3.3 
Resuscitating a baby 3.2 
What are cataracts? 2.6 
What is stress incontinence? 2.5 
Cervical screening 2.0 
What is urge incontinence? 2.0 
What is pain? 1.9 
Type 1 diabetes 1.6 
Healthy eating for diabetics 1.6 
Type 2 diabetes 1.6 
What is coronary heart disease? 15 
Foot care for diabetics 15 
Who is at risk of cervical cancer? 15 
MMR — what parents want to know 1.4 
Diabetes: patient experience 1.4 
Exercise for older people 1.3 
Symptoms of coronary heart disease 13 
All top 20 as percent of total 47 


have a text explanation to these videos and are 
more likely to value the information of these 
videos or at least see it as new information. 
Figures 14-17 look at the relative share of 
video titles within four popular topic 
groupings: diabetes, cervical cancer, 
childhood eczema and stress incontinence. 
There is some visual evidence that videos are 
more widely used by HomeChoice users 
compared to KIT. Certainly for diabetes 
there is a greater relative use for foot care, 
healthy eating and monitoring blood sugar 


Figure 14 Diabetes videos watched in the series 
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KIT 

Top 20 video viewed (ranked) % 

MMR — what parents want to know 74 
Hypertension 6.8 
Type 2 diabetes 5.5 
Getting going 1 47 
What is diabetes? 43 
Prevention 4.1 
Healthy eating 3.5 
Donating blood 2.9 
Treatments for type 1 2.9 
Resuscitating an adult 2.7 
Type 1 diabetes 25 
Patient experience 2.5 
Prostate cancer 2.5 
Moving on 1 25 
Cooling down 1 2.3 
The clinic 2.3 
Diagnose coronary heart disease 2.2 
Symptoms of coronary heart disease 2.2 
Screening for cervical cancer 2.0 
Moving on 2 1.8 
All top 20 as percent of total 67 


Figure 15 Cervical cancer videos watched in the series 
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levels. And use of the two childhood eczema 
videos is more evenly distributed for 
HomeChoice users compared to KIT users. 
However, the percentage distribution over 
videos seems much the same for use of 
cervical cancer videos and stress 
incontinence. 


Videos viewed 
Another way of comparing the services is to 
look at just the number of videos viewed by 
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Figure 16 Childhood eczema videos watched in the series 
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users. Table VI gives the grouped number of 
videos seen by users. 

Of those users who watched a NHS video, 
66 per cent of KIT users and 41 per cent of 
HomeChoice users only watched one video. 
HomeChoice users were more likely to watch 
one or more videos compared to KIT users. 
This reflects a wider use of the videos by 
HomeChoice users. 


Table VI Number of NHS videos viewed by users 








HomeChoice KIT 
Number of videos viewed n % n % 
One only 524 40.6 184 66.4 
2 207 16.0 39 14.1 
3 144 11.1 15 5.4 
4to5 172 13.3 29 10.5 
6 to 10 157 12.2 7 2.5 


Over 11 88 6.8 3 1.1 
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Discussion 


Plainly there was poor take up of videos on KIT 
and this must represent a failure of this aspect of 
the service. Views to NHS videos on KIT were 
approximately one twentieth of that compared 
to the same videos transmitted by HomeChoice. 

The crucial point surely is that 
HomeChoice users did not have a choice 
between using either a text service or a video 
service — they could only view videos. This 
implies that the problems associated with the 
use of videos on KIT were that users 
preferred a text health information resource 
compared to a video information resource. 
Given the choice users will opt for a text 
health service. There is limited evidence to 
support this. In a questionnaire of NHS 
Direct (KIT) users about 60 per cent of users 
said that they preferred the text service, while 
40 per cent preferred a video service. This 
indicates a preference of text over video but 
not a particularly strong preference. Given an 
equal choice between a video and text service 
users would take up the video service about 
40 per cent of the time. However, the actual 
take up on KIT was considerably less than 
this. KIT users on average viewed 60 
information text pages to one video. 

Problems of video use on KIT relate to the 
issues of presentation when two alternative 
ways of receiving information are available on 
the same distribution service. Both a video 
and text service was available on the same 
service and the order on how each 
information media was presented to users is 
important. Here videos were available as a 
submenu of the text service, so that in most 
instances users would have already read 
significant amounts of text on a medical topic 
before they were offered a video to view. 
Hence, possibly, much of their information 
need would have already been met by the text 
service. The video option as an information 
source would not be such an attractive option 
as much of the information the user wanted 
had already been accessed. The low levels of 
video use on KIT does not imply that users 
preferred text more than videos or that the 
videos provided no information, it reflects the 
problems of presenting the same information 
on alternative media on the same service. 


Conclusions 


NHS Direct Digital was available to users in 
Hull and London via two very different 
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transmission DiTV companies. Clearly the 

reaction by users to the same set of NHS 

videos has been very different. As the Wanless 

Report notes “Improving the use of 

information technology (ICT) in the health 

service is a key issue in improving quality and 
productivity” (Wanless, 2002) and the 
comparison here argues that there is still a lot 
to get right. Specifically when videos are 
offered with text on the same distribution 
service, this research indicates that poorly 
thought out menu structure and poor video 
integration with text information will result in 

a poor take up of the video service. The main 

conclusions of the research presented here are: 

°- The reach figure of both services was very 
similar — about 16 per cent. 

* Use on Saturdays and Sundays regularly 
made up between 25-30 per cent of user 
activity. 

¢- KIT attracted a dramatic increase in new 
users in April, which resulted from 
previous users and new users being 
attracted back to the service as a result of 
a one off publicity mail out. 

* New users attracted to NHS Direct 
(HomeChoice) fell by 38 per cent from 
March to May. The number of return 
visitors increased month on month and 
increased by approximately 16 per cent 
between March and May. 

* In terms of NHS videos, 37 per cent of 
use on the HomeChoice and 0.5 per cent 
use on the KIT service related to the 
download of information videos. The less 
than 1 per cent use related to videos on 
KIT represents a failure of this part of the 
KIT service. 

+ The number of NHS videos viewed by 
HomeChoice users was about 200 videos 
per week, however only about ten videos per 
week were downloaded by the KIT users. 

°- The most popular topic videos watched 
for both services were “Diabetes” 
followed by “Coronary Heart Disease”. 

+ The main menu of HomeChoice 
included blood donating as a separate 
item and the greater visibility given to this 
is believed to have encouraged a greater 
use of this video. The video was viewed 
17 times by HomeChoice subscribers for 
each single view by a KIT user. 

* The low take up by KIT users of videos 
seems to result from text content overlap 
in that users would have read related text 
content before accessing the related video. 
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* Of those users who watched a NHS 
video, more KIT users watched just a 
single video: 66 per cent of KIT users 
compared to HomeChoice users — only 
41 per cent of HomeChoice users 
watched one video. 


Notes 


1 An evaluation of pilot projects exploring the health 
applications of digital interactive television, 
Department of Health, 2001-2002. 

2 Conducted by SERCO. 

3 Conducted by City University. 
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Introduction 


ciber’s worx is primarily concerned with 
human behaviour, as elicited by 
bibliometric and latterly computer log 
analysis. Often such studies raise important 
questions relating both to people’s 
negotiation of information systems and their 
reasons/motivations on the one hand, and 
opinions regarding quality, relevance and 
appeal of services delivered on the other. 
Why are certain journals or articles cited 
more than others in particular fields? What 
motivates academic authors to switch from 
book to journal writing? Why do health 
information consumers prefer text over 
video images in some circumstances, or a 
Web site to the personal attention of a 
doctor in others? 

Clearly, these questions can be addressed to 
an extent by surveying users. However, the 
success of ciber may depend on the extent to 
which it embraces multiple methods to 
explore the fundamental issues and questions 
exposed by the generation of statistical data, 
including those associated with the. qualitative 
tradition more commonly associated with 
social science disciplines. 

City University has been funded by the 
Department of Health to undertake a wide 
ranging and in-depth study of the impact of 
the digital delivery of health information 
for the consumer, which has enabled the 
group to deploy a range of methods to 
collect and analyse data and, more 
importantly, to integrate the array of 
methods used. 

Systems under scrutiny for this study 
include touch-screen information kiosks, Web 
sites (“SurgeryDoor” and “NHS Direct 
Online”), and four digital interactive 
television (DiTV) initiatives. Usage, reasons 
for use, attitudes and opinions, and barriers to 
use, are all issues being explored in order to 
provide recommendations to the Department 
of Health regarding how best the public might 
be served information on different aspects of 
health in the future. 

One of the DiTV initiatives, run by 
Kingston Interactive Television in Hull, and 
HomeChoice in London, is a health video-on- 
demand service, which forms part of “NHS 
Direct Digital”. The research reported here is 
concerned specifically with this service, 
reached through the main menu page of 
their TV. 
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Aims and objectives 


The evaluatory sessions were set up to gauge 
opinions about the videos generally, in terms 
of content, usefulness and presentation. 
Specific aims were to examine: 

* usage, reasons for use and how usage fits 
with general information seeking 
behaviour; 

+ how the information obtained was 
utilised, and whether it helped with 
dealing with a health problem and/or with 
the doctor; 

* any barriers to obtaining information 
from the videos, such as terminology, 
poor sound, distracting features etc.; 

« the extent to which video is an 
appropriate medium for the 
dissemination of information, and 
whether some types of information might 
be better presented in other forms, such 
as text or diagrammatically. 


The extent to which findings could be used to 
interpret log results was of major interest. 
Also, as several different groups were 
recruited to evaluate the service (in addition 
to the “real” users or “service user” group), it 
was considered appropriate to determine 
whether, within the sample, there were 
differences in opinion and perspectives 
between the various groups. Of particular 
interest were differences between health 
professionals and the lay public, although 
comparisons between health librarians, 
information professionals and other groups 
were also sought. 


Methodology and sample 


Recruitment and sample 

A number of different groups were recruited 

to view the videos and undertake the 

evaluation. These were: 

* Service users. An online questionnaire, 
activated when subscribers accessed the 
service, asked for interviewee volunteers. 
A total of 12 people offered their contact 
details, but eventually only five were 
willing to participate in the evaluation. 
The real user group consisted of three 
males and two females, aged between 
25 and 60. 

* Elderly people. A group of eight elderly 
people (four men, four women) were 
recruited from the voluntary organisation 
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Third Age (Third Age Project, 2001). No 
attempt was made to obtain people with 
particular demographic or other 
characteristics. Their ages ranged from 70 
to 95, and they viewed a video (on 
chronic heart disease (CHD)) in a group 
at the offices of Third Age. 

Diabetics. Three diabetics were recruited, 
the first being a friend of one of the 
students at City University and the others 
by “snowballing” (i.e. a friend of the first 
interviewee and then a contact of the 
second). All evaluators were females, 
between the ages of 20 and 30, and had 
been diagnosed at least two years before 
they saw the programme. It was not 
possible for these people to see the videos 
together and so individual sessions were 
held with two at City University. The 
third took the video home and was 
interviewed about her evaluation later, at 
her place of work. 

Health professionals. These were recruited 
from City University School of Nursing 
and Midwifery, and consisted of two 
lecturers (one senior) in health and health 
policy who had been nurses, one nurse 
working at Whittington Hospital, and 
four student nurses. The lecturers and 
nurse saw the videos individually; two of 
the students saw them in a wider group 
(consisting of those with a general 
interest, described below) and the other 
had an individual viewing session with the 
researcher. 

Health information professionals. Three 
health information professionals and one 
Master’s degree student working in a 
health environment were recruited, again 
by “snowballing”. 

General interest (i.e. “lay”) volunteers. 
These were people who had no 
professional interest in health, nor a 
particular condition covered by any of the 
videos. They were, rather, simply 
interested individuals, 12 in total, 
recruited via the other interviewees listed 
above. Of these, six were City students, 
one an ex-student, the others being their 
friends or colleagues. Included in the 
total are three friends of the researcher. In 
all, five watched the videos individually at 
home, reporting back later, the others 
with members of the other groups at 
City University. 
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Methodology 

A semi-structured interview approach was 
adopted. For the service users this was looser 
than for other groups, as there might have 
been issues in their usage which had not been 
considered by the researchers when drawing 
up the interview schedule. As they were 
interviewed about videos they had viewed in 
the normal course of their use of the Video 
Networks service, the evaluation was 
retrospective, and included details of the 
accessibility of the system, in terms of menu 
hierarchies etc. With regard to those 
participants who viewed the videos 
specifically for the evaluation, the 
programmes were copied onto standard 
videotape and shown on analogue televisions. 
The service users were interviewed 
individually. Where possible representatives 
from each group watched the programmes 
together, at City University, and discussed 
them in an open forum. Time constraints and 
availability problems meant that some 
evaluators viewed the videos individually, 
with the researcher. In two cases (one of the 
diabetics, and a health professional) videos 
were taken for home viewing, with the 
researcher interviewing the evaluators later, 
also individually. 

Before the showings (and lending), 
participants were told a little about the 
research project and about the DiTV service 
from which the videos had been taken. (The 
actual service users had, of course, already 
seen the videos beforehand, for their own 
benefit.) An outline was given of the issues to 
be discussed after the showing. An evaluation 
form had been drawn up to reflect the aims of 
the exercise, and this was referred to but not 
looked at in detail until the viewers had watch 
the first section of the video chosen. This was 
so that they would have an idea of content 
and presentation etc. before needing to 
consider these and other aspects in detail, as 
outlined on the form. In every case, even 
where the researcher was present at the 
showing and had the opportunity to discuss 
the programme “live”, volunteers completed 
the evaluation form (one for every video 
viewed). This both provided a structure for 
the discussion and a written record of views. 

The service user group was in a position to 
provide information on all the following 
topics, which were not appropriate to those 
viewing the programmes especially for 
the evaluation: 
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+ Reasons: for turning to the service. 

* Context of use: how use of the service 
related to wider health information 
seeking behaviour. 

* Relevance: the relevance and usefulness of 
the information regarding the particular 
circumstances for which it was sought. 

* Uses: how the information was used, and 
any benefits that accrued {such as being 
better equipped to cope with a medical 
problem or deal with a medical 
professional. 

* Usability of the service: the ease with which 
the videos were accessed from the menu 
hierarchy. 


The areas discussed by all participants (i.e. 

the service users and the other groups) were: 

¢ Information content: the 
comprehensiveness and appeal of the 
information and, in its broader sense, its 
potential relevance. 

e Presentation: order of scenes, production, 
performance of doctors/lay participants. 

* Language and terminology: medical 
language, descriptions of procedures etc. 

* Format: appropriateness of the topic for 
the medium, and whether any of the 
content could have been better presented 
in alternative forms. 


Evaluators could raise any points not covered 
above, either in person or on the evaluation 
form. 


The videos 

The service users, of course, had seen any one 

or more of the videos, and were asked for their 

views and reactions to the ones they had seen. 

For the other groups, a selection was chosen 

for viewing that reflected the range available 

and type available. After viewing the majority 

of the videos, it was possible to informally 

classify them to the extent necessary to choose 

a representative sample for evaluation. The 

categories, with the videos subsequently 

chosen, were: 

* Specific conditions (CHD, diabetes). 

* General condition management (pain 
management). 

* Procedural information (giving blood). 

* Practical information/demonstrations for 
specific demographic groups (exercise for 
older people). 


The videos with predominantly medical 
content (CHD, Diabetes and Pain 
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Management) had a similar production style. 
A GP or similar expert spoke to camera or, at 
times, voiced over a specific scene (such as a 
consultation). The videos were broken down 
into standard sections (accessible 
individually), such as “Symptoms” 
“Medication”, and “Patient experiences”. 
The latter consists of individual patients 
describing their experiences to camera. The 
Giving Blood programme is very different. It is 
not split into sections and also, unlike the 
medical conditions programmes, it follows a 
person through a procedure — it is 
chronological, and it places the patient as the 
central focus. Finally, the Exercise for the 
Elderly video (and others, such as First Aid) 
consists of demonstration activities. 


Results 


Usage issues 

Only the service user group were asked about 
issues related to usage, as those of other 
groups viewed the videos especially for the 
evaluation. 


Videos watched and reasons for viewing 

As the service being evaluated was available for 
only three months, it is perhaps not surprising 
that general curiosity about the service and a 
general interest in a particular topic were the 
principal reasons given for watching the videos 
rather than for a specific medical need that may 
have arisen whilst the service was operational. 
One described her viewing as being of “a clip 
here and there” to get a general flavour of the 
service and for future reference. Others also 
found any topic that they had a general interest 
in (such as healthy eating) and viewed out of 
general curiosity. One person did watch a video 
based on her condition, choosing the 
osteoporosis and diabetes programmes, both 
because she is a sufferer. Interestingly, another 
respondent also looked for particular topics 
based on a current health need — sleeping 
disorders and restless leg syndrome — but 
ended up watching a video on first aid instead, 
as there was no material on those topics. She 
has two teenage children, and felt that knowing 
first aid might prove useful in the future. The 


. other service users also confirmed that they 


would watch any of the videos if an information 
need arose which was covered in the series. 
These results reflect log findings, but only 
tentatively. These show that videos on “major” 
conditions were consulted more (the most 
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viewed being Diabetes, CHD and Foray for 
Health — a general healthy living video). Topics 
that may affect fewer people, such as 
childhood eczema and cryoanalgesia were not 
widely consulted. However, the video on 
“stress and relaxation”, which might be 
considered to be of wide general appeal, did 
not attract a single viewer. A general point is 
that, clearly, the extent of viewing out of 
“general interest” and, by contrast, as an aid to 
manage a particular condition requires further 
research, possibly using survey methods. 


Integration of video service with other information 
sources 

For this small sample it appears that, as with 
qualitative research for other strands of the 
same research programme, users of the DiTV 
service were also big users of other sources — 
particularly the Internet (Nicholas et al., 
2003). Interestingly, this is not borne out by 
quantitative results, which appear to indicate 
that DiTV reaches audiences who are not 
users of other information systems (Nicholas 
et al., 2002) and so may be attributable to 
heavy information users being more disposed 
to discuss their information seeking habits. 
Four of the five were Internet users (which, 
for one, meant booking a terminal in his local 
library) and searched for health information 
from this source. Indeed, the respondent with 
restless leg syndrome claimed to have 
researched her condition on the Internet and 
presented her findings to her doctor, telling 
him “this is me”. She added: 


If I didn’t have access to the Internet, then I 
would never find out about my true condition”. 


As with other respondents, she said she 
“like[s] information in general” and has “lots 
of reference books and lots of alternative 
medicine books that I tried to go to first. 

The non-Internet user was nevertheless an 
avid information user. She said she consulted 
books first, “because I am such a techno- 
phobia [sic]”. She added: 


... if the TV could offer me information 
adequately, then I would go for the TV first, 
whether it’s in text or video form because it’s so 
familiar to me and my kids to use the remote 
control. I am beginning to learn that I don’t have 
to carry books around to get information. 


The Internet users also saw the TV generally 
as a good source of information for health 
topics. 

The DiTV service, as often with the 
Internet, appears to be principally the first 
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port of call. Indeed, the service has 
supplanted the Web as the initial point of 
enquiry for some people: 


I have only browsed the HomeChoice service, 
but feel it would fit in very well with my Internet 
use. In fact, if the topic I was interested in were 
on the DiTV service I would check that out 
before logging on to my Internet account. 


And: 


I would look it up first, see if it has something 
that I am looking for and watch it. Then I will 
probably look it up on the net further for 

some more info or even different opinions on the 
same topic. 


In its role as the first point of contact the 
DiTV service: 


... triggers off my interest. It makes me want to 
go and read about it elsewhere. ... it means I can 
go to the library or on the Internet and I can 
elaborate on it even more; 


Similarly: 
I access it initially for first hand information and 
then I can go on from there ... it feels a lot easier 
in the comfort of your own home to watch some 
video than trying to search some written text 
about certain health issues ... [it gives] me 
further contact points as to where to go and what 
to do next. 


A major question for the overall project is to 
examine the extent to which information 
seeking in a health context is undertaken on 
behalf of other people. Possibly due to the 
brevity of service availability and relatively 
limited range of videos available, none of the 
respondents needed to use the videos as an 
information source on behalf of their family. 
However, much researching for information in 
the health field was said to be undertaken for 
others. One respondent undertakes a substantial 
amount of voluntary work which involves trying 
to understand different illnesses and conditions, 
and uses books, NHS Direct phone line and 
various Internet sites to find answers to 
questions; and other interviewees gave various 
stories of family problems. There was also praise 
for video-on-demand services for their potential 
to reach people who might be too embarrassed 
to seek information from others: 


It will be very good for them to find out about 
recreational drugs, also it will be useful accessing 
information which they might find embarrassing 
to ask for in general like sexually transmitted 
diseases, HIV and AIDS, contraception, issues 
most young people would like to know. 


Use of the information 
For most interviewees, the principal use of the 
service was to provide background 
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information to explore in greater detail 
elsewhere, as outlined above. It was 
mentioned that the information to be found 
on the video service could be used as a 
substitute for seeing the doctor: 


I got some idea about my condition through the 
video. You can’t discuss with your doctors as to 
all the ins and outs about the condition, it 
prompts me then to get onto the Internet and 
found more information about it, like the 
osteoporosis society’s detail and I got in touch 
with them. 


There was also some indication that the 
service could be useful in viewers’ dealings 
with the doctor. One respondent claimed to 
always prepare a list of questions to ask during 
a consultation, derived from personal 
research, and said the video-on-demand 
service would be most useful here. Another 
confirmed that: 


... it just gives you more confidence in asking for 
the right thing. It helps in getting what you need 
really ... It gives you the knowledge in asking 
what you really need. 


A third felt that it was difficult to obtain 
enough information from doctors because 
they did not have time to go into details, and 
that starting from a position of some 
knowledge would obviously be beneficial in a 
situation of tight time constraints: 


If I understand my condition better, the doctor 
can spend more time with me on how to treat my 
condition better. 


An alternative view was that “doctors don’t 
like it when you know a little about it. They 
like it even less when you know a lot”. Despite 
this apparent antagonism it was still worth 
acquiring information as, “in the end they 
give in and listen to you”. A different aspect of 
dealing with medical professionals was 
outlined by one respondent who watched the 
video about NHS Direct Helpline and then 
promptly used it to call about an infection 
suffered by her son: 


I had heard about it [the NHS Direct telephone 
line] but the programme explained it so I knew 
what to expect. 


He confirmed that he would not have used 
the telephone service without this prior 
knowledge. 

The extent to which the video programmes 
may affect health related behaviour is difficult 
to determine. One interviewee claimed that 
the videos on healthy eating and healthy living 
generally had stimulated her into buying more 
fresh fruit and vegetables. However, the fact 
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that these were the topics she chose to watch 
indicates an interest in the topic and possibly 
a predisposition towards improving her diet 
anyway. It is, of course, possible that a desire 
is only fulfilled with a particular stimulant 
such as the depiction of healthy foods or a 
good workout, but actually determining a 
causal relationship is fraught with difficulties. 
Suffice here to say that self-report data from a 
small sample interviewed in-depth appears to 
suggest that under certain circumstances, a 
television programme may influence health 
related behaviour. 


Usability of the service 

The final area only appropriate to the service 
user group was that of the usability of the 
service. Other participants could not 
comment on this as they viewed the videos on 
an analogue TV via a standard video-recorder 
and did not have to negotiate a menu 
structure or use a remote control. Service 
users, on the other hand, had been exposed to 
the full system, and they were asked how easy 
they found it to access and use. No problems 
were reported with the navigation of the 
system, variously described as “simple”, 
“easy” and “straightforward”. As mentioned 
above, the service was accessed through the 
“LeisureChoice” option on the main menu, 
with NHS Direct being the required option 
from the sub-menu, which also contained 
“Films”, “News”, and “Sport”. Thus, 
activating only one menu level was enough to 
arrive at the main page of the health 
information material. The only issue raised 
was that some people might not equate 
“Leisure” (as in “LeisureChoice”) with health 
information. One respondent felt that NHS 
Direct was important enough to actually be 
one of the main menu items, and another felt 
that “LeisureChoice” should have been 
re-labelled “Lifestyle and health”. 

The facilities to pause, fast forward, stop 
and play again at leisure were very well 
received indeed: “because I work from home, 
if the phone rings, I can just pause it and 
continue later. I skip forward a lot, if I know 
certain parts, I often do that, which is a great 
feature of the service”. 

Having the videos split into small 
subsections was also favourably received, as it 
was said to help people to target information 
without viewing extraneous material. This 
aspect was exposed to all participants. Only 
the health professionals questioned it, with 
two evaluators indicating that watching the 
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programmes in “any order” might be 
confusing for viewers, and make it harder to 
assimilate information. 

All participants were able to comment on 
the following aspects of the service: 
information content, presentation, language 
and terminology, and format. Results are 
outlined in the sections below. 


Information content 

The service users were generally positive in 
their appraisal of the video content. The 
respondent who browsed through several 
declared that “generally they were very 
informative, not patronising and would be 
useful to go back to”, and another said of the 
diabetics video, that “the information was 
good — what I would want to know”. The 
osteoporosis video viewer said that it “made 
me aware of what I had, the reason that I had 
it, and what the prognosis is like and how to 
slow down the progression of the disease”. 
Although one interviewee complained that 
the programme, at 15 minutes, required quite 
a bit of concentration (in fact, none of the 
individual clips last more than five minutes), 
other interviewees felt that, in fact, there was 
not enough information included. One 
respondent even suggested an option to 
access more clips treating the same topic but 
in greater depth: 


For example, it could have a couple of more clips 
on the latest research on the subject which it is 
talking about and new scientific discoveries or 
something similar, or maybe even some 
controversies involved in researching the subject 
concerned. 


The groups viewing the videos for evaluative 
purposes only were also generally positive 
about programme content. The general 
interest evaluators said that the information 
was generally “comprehensive”, for both the 
Diabetes and CHD videos. However, the 
medical and information professionals felt that 
these videos “tried to cover too much ground 
too quickly”. There was “too much to take in”, 
and many topics were “poorly explained”. One 
of the medical professionals felt that the subject 
matter was “difficult”, and the conditions, 
particularly CHD, “complex”. The differences 
in opinions about the videos between medical 
professionals and other evaluators is 
interesting, and is explored in detail in the 
discussion section later in this report. 

The elderly group saw only the video on 
CHD. Their views are valuable, partly 


Home electronic health information for the consumer 


Aslib Proceedings: New Information Perspectives 





Peter Williams, David Nicholas and Paul Huntington 


because they saw the programme without 
members of any other group being present, 
partly because there were more of them to 
provide input (eight), but mainly because they 
are a group targeted by the Department of 
Health as requiring (digital) health 
information. The Wanless Report (Wanless, 
2002) makes much of the fact that the 
population is ageing, and that the high 
number of elderly people in the future will 
place a strain on the NHS that could be 
reduced by higher engagement with and 
responsibility for their own health. The group 
was very impressed with the content of the 
CHD video, describing it as “marvellous”, 
“really informative” and “a good thing to help 
people with the condition”. They accepted 
that they were a group that needed 
information on coronary problems. 

It is worth discussing the video on diabetes 
in some detail, as it was possible to recruit for 
the evaluation those who actually had the 
condition, providing the perspective of those 
at the front line and for whom the information 
was written and health professionals with an 
intimate knowledge of the condition. There 
were, in fact, interesting contrasts between 
the views of these two groups. The former 
were far more positive. The diabetics felt that 
the content, as one put it, “was all very 
relevant and covered the most important 
points”. The other diabetics also indicated 
they were happy with the content. The 
general viewers and information professionals 
also considered the content “relevant”, 
“comprehensive” and “interesting”. One of 
the medical professionals (a nurse), however, 
suggested that much of the material was not 
very clear and that some of it was too detailed. 
Another said there was, at 22 minutes, too 
much content. There were, from other 
quarters — the diabetics and an information 
professional — suggestions for the inclusion of 
additional material. One pointed out that the 
“important topic” of genetic inheritance was 
not covered. Another considered that it would 
have been helpful to have had a section on 
how a newly diagnosed patient should expect 
to be treated by the health service, in terms of 
engagement with surgeries, clinics etc. and 
the roles that might be expected of GPs, 
nurses and others. Finally, one of the 
diabetics argued that there should be 
something included about the psychology and 
mental/psychiatric implications of being 
diagnosed with the condition, or “at least” the 
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social implications and how to deal with 
them. There were also questions that the 
content raised and appeared not to answer. 
These included why diabetes causes eye/ 
heart/kidney damage and why fat needs to be 
controlled as well as sugar. 

The Giving Blood video, made in a different 
style to the others of a purely medical content, 
was very positively regarded by all the 
evaluators. The content was considered 
“interesting” and “appropriate”, with the 
procedure described in an easy to understand 
and logical order. There were, however, 
(almost inevitably) suggestions for additional 
information to be provided. This was 
principally with regard to the benefits a blood 
bank gave to a hospital in treating patients 
and the value, therefore, of giving blood. As 
these benefits were not emphasised, in the 
opinion of the evaluators, the video was 
described as appearing to aim at the “already 
committed”. It was felt that the programme 
should have aimed more at recruiting donors. 
Highlighting the benefits one gave to others 
would, it was felt, encourage the activity. As a 
footnote to this point it may be said that as the 
videos are “on demand” those electing to view 
this particular one would already have 
considered the possibility of becoming a 
donor — just as those watching a programme 
on healthy eating may be already thinking 
about some action on their part. This is unlike 
the case where a broadcast programme may 
reach people who are watching by chance and 
get the idea of giving blood specifically from 
the programme. 


Presentation 

There were mixed views with regard to the 
presentational aspects. The presenters (i.e. 
doctors) were criticised as being “boring” and 
“uninspiring”, with one viewer, a nurse, 
complaining “they reinforced the stereotype”. 
When asked to expand she said they were 
“aloof, distant and unfeeling”. Members of 
the elderly group felt that the doctor featured 
was “typical” in this respect, although others 
said that their own doctors were friendlier. 
The video was redeemed by the participation 
of patients (who appear in a section entitled 
“Patient experiences”). These were described 
as “genuine”, “warm” and “interesting”. 
Given this, it is perhaps surprising that no one 
considered that there should have been 
greater participation by the patients. As with 
the case of Diabetes and Pain Management, 


Home electronic health information for the consumer 


Aslib Proceedings: New Information Perspectives 





Peter Williams, David Nicholas and Paul Huntington 


however, there was a general feeling that 
instead of “lumping together” the patients in 
one section, they should feature at various 
points. For example, when the doctor 
describes the symptoms, it was considered 
better to have had a sufferer reflecting the 
professionals’ comments by talking of his/her 
own situation. In the video shown, the doctor 
is a voice-over in a scene showing a man 
holding his chest and staggering. This was 
regarded as “distracting” at worse, and 
“unhelpful” and “unconvincing” at best. 

Several viewers from the service users, the 
general interest and health information 
professionals groups thought that too much 
attention was made of the speakers and not 
enough on using the video medium to look at 
procedures, situations etc. As one of the 
nurses said, “the possibilities are there, but 
most of the programme was a guy talking”. 

One point of interest regarding the 
production and presentation of the videos was 
opinions about the involvement of the general 
public/patients in the programmes. 

The Pain Management, CHD and Diabetes 
videos all lead on health professionals, 
generally as “talking heads” explaining 
medication, processes etc. Public/patient 
participation is somewhat minimal, except in 
separate sections entitled “Patient 
experiences”, where patients also appear as 
“talking heads”, describing their own 
symptoms and treatment etc. 

Evaluators considered that although there 
was sufficient involvement of the public in 
these videos the type of participation was not 
ideal. Nearly all evaluators, including the 
service users, felt that there should have been 
a far more dynamic involvement: “The 
patient is just there like a tailor’s dummy”. 
The service user who watched the programme 
on diabetes remarked, “that bit about diet — 
why didn’t they show a family shopping or 
planning a meal, with the heart-attack guy 
saying something like “you’ve got to buy extra 
fruit — the doctor said it keeps the fat down”. 

Illustrating the appeal of having the public 
more engaged in the programme content was 
the positive view of the programme on giving 
blood. This video followed a potential blood 
donor through the blood giving process. The 
focus was thus on the lay member of public, 
with the medical professionals reacting to 
him, rather than the “talking head” of the 
doctors, with the patient a secondary figure, 
as in the CHD and other medically oriented 
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videos. The medical professionals appeared as 
they would in the real situation (i.e. to register 
him, take a sample etc.). Evaluators from all 
groups viewing this video felt that this was an 
excellent approach. 

Interestingly, the diabetic group did not 
regard the degree or type of patient 
involvement as an issue. They felt that the 
split of medics in some sections and patients 
in others was “well done”, and were 
comfortable with the practice of having 
patients explain their condition in a separate 
section. One said that this did not interest her, 
as she knew her own experiences, and was 
more interested in latest developments in 
treatment, rather than the views of other 
sufferers. It might have been interesting to 
have shown the video to a group of newly 
diagnosed diabetics, who might have had a 
different outlook. 

One other point was made with regard to 
the video participants. One of the health 
professionals described them as “all white and 
middle class”. Apart from this lack of 
inclusivity, she pointed out that much 
diabetes occurs in the Asian and 
Afro-Caribbean communities. In addition to 
the lack of contributors from these groups, the 
information given in the programme did not 
allow for differences in diet or lifestyle, which 
affect the condition and its treatment. The 
diabetic evaluators did not mention this 
aspect. This is probably due to the fact that 
they themselves were all white. Also, it may be 
that the health professionals may know more 
about the condition than the sufferers, 
certainly in the field of epidemiology. The 
general interest viewers (one of whom was of 
African origin) similarly did not comment on 
this, possibly not knowing the ethnic bias the 
condition manifests. 


Language and terminology 

Evaluators’ views on the language used in the 
programme were mixed, although within- 
group opinions tended to be similar. The 
health professionals, in general, considered 
the language to be too difficult and were 
somewhat critical of the language. The elderly 
group also felt there were many medical terms 
and expressions that they did not understand, 
but were quite accepting of this. Other 
viewers, including one of the health 
information professionals, thought the 
language was generally understandable, one 
going so far as to say that terms used were 
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“well explained”. The elderly group had an 
interesting view of this aspect of the 
programme. They claimed that despite 
difficult terms, the language used was not a 
problem. One reasoned, to general 
agreement, that “there is so much around 
today that we don’t understand — we just let it 
go by!” Pressed on this point, others said that 
there were so many new developments they 
could not keep up with that they tried not to 
think about it. One said his video-recorder 
was the oldest and simplest available, because 
the instruction book for new ones was 
“ridiculously complicated”. The video viewed 
(CHD) was beneficial, it was claimed, as it 
was not difficult to grasp the essential 
messages (in this case regarding diet, lifestyle 
and treatment etc.) even without a command 
of some of the vocabulary. Clearly, visual 
material is appropriate for those who have 
doubts about technical or medical vocabulary. 
Much material, however, consists of a 
“talking head”, where it could be argued that 
the medium is not being exploited to its full 
extent. Despite assurances that the video was 
“interesting” and “helpful”, it remains for 
further research to determine the amount and 
value of information that is actually imparted. 

The diabetics felt the language was 
“pitched just right” with regard to the video 
on their condition. A nurse who had originally 
criticised the language later said she felt that 
long-term diabetics would probably be 
familiar with the terminology, but newly 
diagnosed cases and carers would not, and 
opined that the video would be for these 
people too. The terminology used in the 
Giving Blood programme was considered very 
understandable by all groups. Unlike with 
others dealing with medical (as opposed to 
general health) issues, there were no 
complaints by the medical professional 
viewers or others about difficult technical or 
medical terms. This may have been because 
the information was presented within a 
specific and demonstrated (i.e. “acted-out”) 
context, although evaluators did not consider 
this possibility. Any deeper analysis would 
have to work through the programme 
transcripts, run them through a reading level 
programme and then compare results with 
those found by interviewing viewers. 


Format 
There was much agreement amongst 
evaluators of all groups with regard to 
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whether some (or all) of the information 
would be better presented in another form. 
Particular emphasis was placed on text, but 
viewers were also asked about diagrams or 
animation. People from every group felt that 
the sections on medication, in which a doctor 
talks about various medicines and their 
effects, could have been better presented as a 
table, listing medications, functions, effects 
etc. It was variously suggested that this could 
be on screen with a voice-over giving 
additional details, accessible from a remote 
control button, whereby viewers could toggle 
between the video and the table, and, by 
contrast, as a replacement for the video 
section. Many people felt that, without such 
textual reinforcement, this section was 
ineffectual: “No-one could remember all 
those details”, “You would have to be a quick 
writer to get all that down for when you 
needed it!” It was suggested by two of the 
health professionals that the classification of 
the medication (i.e. curatives, pain relievers 
etc.) was not explicit enough, and that a table 
would have been of much help here. Others 
(two health information professionals and a 
general interest viewer) felt that a written 
summary of every section would have been 
helpful. Sections on diet (in CHD, Diabetes, as 
well as in the Healthy Eating video), could, it 
was suggested, also benefit from recipes 
displayed on screen and, ideally, printable. 

Evaluators’ preferences for material in 
written form is backed by research showing 
this to be beneficial. Studies have shown (e.g. 
Kitching, 1990) that generally patients forget 
half the oral information provided to them 
within five minutes of leaving a consultation. 
There is no reason to suppose their ability to 
retain such information from the television 
may be any better (although in this case there 
is, of course, the option to replay the tape). 
Studies in the USA (e.g. Wehby and Brenner, 
1997) have shown that patients, in this case 
with chronic heart failure, find it difficult to 
retain information and that they may not 
appreciate the relevance of information 
provided by clinicians. Therefore, written 
advice, particularly if printable, would seem 
to be a good option. 

A possible preference for text services is 
reflected in the logs. In total, 85 per cent of 
users did not view a topic video, though these 
users made use of accompanying textual 
information, and 89 per cent of user sessions 
did not include a view of a topic video. In the 
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paper by Huntington et al. (2003) in this 
issue, striking differences are evidenced in the 
extent of take-up of this service — which 
provides video information accompanied by 
text-based materials, and that of a similar 
service (indeed, the videos are the same) in 
which only the videos are offered. The video- 
only service had a greatly reduced reach figure 
(i.e. the number of people who used the 
service compared to the number to whom it 
was available). 


General discussion and conclusions 


A number of issues were raised in this 
exercise, and several important points 
emerged. Of particular interest was the fact 
that there were several instances where the 
general interest evaluation participants and 
the service users appeared to be more positive 
about the videos than either the health 
librarians or, more particularly, the health 
professionals. This was most marked in the 
videos about specific conditions (e.g. 
diabetes, CHD). The differences were not so 
marked in the case of general health videos 
(Healthy Eating) There may be several reasons 
for this: 

+ The health professionals and, to a lesser 
extent, health librarians, have a better 
awareness and knowledge of the subject 
and are more able to judge the accuracy 
and comprehensiveness of content and, 
thus, be more critical. A good example of 
this was the evaluation of the Diabetes 
video. One of the nurses pointed out that: 
“much diabetes occurs in the Asian and 
Afro-Caribbean communities. There 
were no contributors from these groups, 
and the information given in the 
programme did not allow for differences 
in diet or lifestyle, which affect the 
condition and its treatment”. As 
mentioned above, other viewers might 
not have known about this important 
omission, and so judged the programme 
more positively. 

* Lay viewers look at the videos more in 
terms of a “need to know” perspective, 
and may filter unnecessary information, 
including difficult terminology. This was 
particularly apparent in the case of elderly 
people, who admitted their ignorance of 
medical vocabulary but claimed to have 
understood the main messages from the 
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videos: “of course, I didn’t understand 
every word, but you don’t need to — I got 
the gist — more than that really”. 

+ It is possible that the medical (and 
information) professionals underestimate 
the ability of lay people to understand 
both medical terminology and concepts. 
It was always medical professionals (and 
some members of the elderly group) who 
claimed the language was too difficult. 

+ There may be an element of “territory 
marking” going on, whereby the medical 
professionals are seeking to hold on to 
their status as knowledgeable guardians of 
health information. The ever-widening 
availability of health and medical 
information for the “consumer” may be 
seen as a threat rather than a help. It 
might be natural for doctors and nurses to 
dismiss much medical information as too 
difficult for the lay user. 

¢ Alternatively, it may be that those lay 
viewers who said that the level of 
language was not difficult were reluctant 
to admit ignorance. It is hoped later in the 
project to carry out tests, which would 
examine information reception and 
retention. This will provide a more 
accurate assessment of language and 
terminology than self report opinion data. 


An interesting point, that may be explored in 
more depth in the future, is the possible 
differences in content, presentation and even 
format which people watching the videos 
from different perspectives might benefit 
from. In the diabetes programme, for 
example, the diabetics themselves were happy 
to listen to a “talking head”. As people 
experienced in coping with the condition, 
they were not looking for demonstrations of 
procedures or depictions of fellow sufferers 
going about their daily lives. They wanted 
more in-depth information, on new 
medications etc. Those with a general 
interest, however, felt that the programme 
was lacking and wanted a more dynamic 
production. This may be because they have 
less at stake, as regards information 
acquisition, and therefore need to be 
entertained more. Viewers from different 
ethnic backgrounds may need information 
that is tailored to their lifestyle, diet or 
culture. It may even be that, if videos are to be 
translated, the resulting text is not simply a 
word-for-word copy of the original, but 
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modified to suit particular sensibilities and 
issues. Clearly, having short video-clips of 
between five to ten minutes, on-demand 
facilitates customised content (indeed, this 
has even been undertaken to an extent — the 
programme on CHD includes a “Patient 
experiences” section repeated using sign 
language, and again with subtitles). 

A related point is the material respondents 
considered better presented in text form. This 
was principally the sections in the Pain 
Management and CHD programmes. In both 
cases there was a strong feeling that the 
information on medication would have been 
better presented in text, as a list or, more 
popularly, in tabular form. This view may 
represent an early indication of the type of 
information people would prefer in text form. 
It seems to indicate that this is where the 
information is of the type requiring repeat 
access, such as a series of simple facts (in this 
case, reasons for taking the medication, how it 
helps, and what the side effects are). 

With regard to the latter point, it is worth 
mentioning that there is much log evidence to 
show that text services are far more used than 
video. Despite the programmes being divided 
into a number of small units, it is more 
difficult to pinpoint a particular item or fact 
required from a video than from text. The 
former cannot be browsed or scanned in the 
same way — only “serial access” is possible. 
For this reason it might be that video is more 
appropriate for general background 
information, enabling an overview of the 
topic. Experiences and opinions or other 
“general interest” type material not required 
for consultative purposes might also be well 
presented in video form. 

A very strong finding emerged regarding the 
participation of patients themselves. There 
was universal agreement that the amount of 
time they were allocated was sufficient, but 
that what was really wanted was a more 
dynamic positioning. where there was more 
interplay between patients and the health 
professionals. Viewers clearly wanted a heavy 
involvement by the professionals — the whole 
idea of the programmes was to provide 
information and advice — but they also wanted 
the perspective to be that of the patient. The 
generally unfavourable comments regarding 
the performances of certain of the health 
professionals (that they were “aloof” and 
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“anfriendly”) might have been ameliorated 
had there been more footage of interaction 
between patients and professionals. It is 
interesting that the sections offered in the 
videos in which patients (as “talking heads”) 
appeared were not the most commonly 
viewed pages, as elicited by the logs of service 
use. This emphasises the importance of 
integrating this aspect of the topic with 
medical information, and not having it as a 
divorced bolt-on. 

In conclusion it may be said that although 
the idea of informing the public about health 
by using the medium of video may appear 
attractive, there are many issues to address. 
Not least of these is the type of information 
that may be appropriate to mediate in this 
form. Problems of access to specific 
information points and the need to record 
information (i.e. such as that on medication) 
indicate that a moving image might not be the 
best form of representing some information. 
Secondly, the “talking head” approach does 
not appear to be the best way of exploiting the 
medium. 
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Introduction 


A paper (Warner, 1997) traced the public 
reception of the Research Assessment 
Exercise 1996 (RAE) and was effectively a 
modern social epistemological study, 
focussing on social rather than primarily 
individual ways of knowing. The concerns of 
that paper with research evaluation, with 
formulating a communication model, with 
demonstrating the explicability of information 
transfer, and with the current possibilities for 
social epistemology have all been 
subsequently developed. These developments 
are considered here. 

The venue (The Swan Hotel, Bedford) for 
the conference at which a version of this paper 
was delivered is associated with Jobn Bunyan. 
Some of the allusions connected with the time 
and place of oral utterance retain relevance in 
written form. For instance, the paper was __ 
considered to be given in the existential 
interval discovered in “the midst of a game of 
cat”, recalling Bunyan’s autobiographical 
experience recounted in Grace Abounding to 
the Chief of Sinners: 


... and having struck it [the cat] one blow from 
the hole; just as I was about to strike it the 
second time, a voice did suddenly dart from 
heaven into my soul, which said, Wilt thou leave 
they sins, and go to heaven? Or have thy sins, 
and go to hell? (Bunyan, 1987a, §.22). 


The voice heard by Bunyan was real in its 
effect (the beginning of Bunyan’s process of 
conversion). Analogously, the theme that 
perceptions are real in their consequences, 
traceable to sociology in the 1930s (although 
there are other antecedents), lends 
significance to the social epistemological 
studies which are to be described. In ordinary 
understandings, too, there would be 
agreement on the significance on the mass 
media, including newspapers (exoteric media 
in the classification to be developed here) to 
public perception of events and to policy 
formation. A puritan conscience could be 
detected in the focus on the individual 
intellectual labour of the researcher, although 
research also includes elements of deliberately 
collaborative intellectual labour, particularly 
dialogue with students. 

A collaborative impulse is also revealed in a 
deliberate inheritance from structuralism. 





The author would like to acknowledge the 
collaborative intellectual labour of the 1997 and 
2001 student cohorts. 
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Following Levi-Strauss, the thinking subject 
is disregarded and the social epistemological 
study proceeds as if thinking was taking place 
in myths or texts. The concern is not with the 
personal author of an article but with the 
place of contributions within the social 
production and communication of 
knowledge. Studies produced on these 
assumptions may not have immediate 
application (contrast the proposal to 
substitute citation based measures for direct 
peer evaluation of research) but can have wide 
and significant implication. 


Research development 


The paper, “The public reception of the 
Research Assessment Exercise 1996” 
(Warner, 1997), which is the proximate origin 
of the themes to be discussed, was first 
developed informally, by the personal 
collection of newspaper cuttings. Research 
was then progressed more formally (although 
without specific funding) by searching 
newspaper and magazine databases and by 
analysis of retrieved material, using 
techniques derived from grounded research. 
The paper was written shortly after the 
communication events it studies, giving it 
unusual currency in academic terms. 

The topic reflects a concern with the 
position of academics, who, as personal 
subjects, communicate with others who find 
the nature of their work remote and have 
illusions of its tranquillity. Two aspects to the 
study can be distinguished: 

(1) An empirical focus (the reception of the 
Research Assessment Exercise 1996). 

(2) A communication model (supported by 
information retrieval techniques). 


The communication model, and the 
possibilities offered by information retrieval 
techniques and resources, are probably of 
greater interest, with stronger possibilities for 
development, than the specific empirical 
focus. 

I will not directly review either the empirical 
focus or the communication model embodied 
in the original paper, but will consider their 
subsequent development and anticipate further 
possibilities. The diverse intellectual labour 
represented by the spoken contributions of the 
conference participants will now be 
incorporated into the written paper. 
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Empirical focus 


The public reception of the RAE 1996 
formed the empirical focus of Warner (1997), 
with the level of granularity largely stopping 
short of intra-disciplinary discussions. Some 
attention, however, was given to the literature 
of information science, particularly to 
previous contributions which had been 
concerned with citation analysis and the 
assessment of research. These contributions 
were mentioned antithetically, suggesting that 
there had been a failure to accept 
responsibility to inform discussions of 
research assessment from understandings of 
scholarly communication and the 
development of knowledge. 

A collaborative approach, incorporating 
spoken and written interaction with science 
communication students, led to further 
developments[1]. The primary focus of 
student responses was not technical (although 
significant issues were raised) and many 
suggested that the approaches could be 
complementary rather than antithetical. 
Crucial points relating to research evaluation, 
emerging from the dialogue with students, 
were synthesized in Warner (2000a). 


Description and value 

A distinction between description and value 
was not necessarily fully, if at all, made in 
discussions of citation counts and research 
evaluation. By description, is understood the 
correlations established and by value, the use 
made of those correlations. For instance, 
should we use citation data to determine or to 
inform our judgment? 

Although a distinction between description 
and value is not fully made in the studies 
which have proposed substituting citation 
measures for direct peer-based research 
evaluation, it emerges indirectly in an area 
which mediates between description and 
value (this could be regarded as the return of 
the repressed). The issue of the 
transformation of citation rankings into RAE 
grades begins at the level of description but 
modulates into considerations of value. At the 
technical or descriptive level, the correlation 
demonstrated by studies is between rankings 
for Units of Assessment (UoA) by citation 
counts and by Research Assessment Exercise 
grades, not directly between the numbers in 
each category (early studies obscured this 
distinction). At the level of value, the 
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distinction is forcibly brought home when an 
attempt is made to translate from citation 
counts into possible research grades, possibly 
for the purposes of prediction, and the 
complexity and contrivance of the procedure 
is exposed. 

Macro- and micro-levels of correlation can 
be distinguished. Macro is understood as a 
collectivity of institutions (which might be a 
UoA) and micro as the individual entity 
within a collectivity, such as an academic 
department. At the macro level a strong 
correlation exists between rankings derived 
from citation counts and from RAE grades. 
Correlation at the micro level is far more 
variable. These correlations also correspond 
to the level of description. Dialectical progress 
can be made by restoring the distinction 
between description and value and examining 
more carefully what exists at the level of 
description. 

In other contexts, concern has been 
expressed at the relative lack of validating 
studies for citation analysis (Garfield, 1998) 
(there is an analogy here with debates in 
information retrieval system evaluation where 
the possible distortion of relevance judgments 
by their experimental contrivance, compared 
to real world relevance considerations, has 
been increasingly recognised). The RAE 
grades represent a historically rare instance of 
real world, and not experimentally elicited or 
contrived, judgments of research quality, which 
lend themselves to simple rankings. The 
studies can then be regarded as yielding a 
significant validation of the correlation between 
citation counts and other intersubjective 
methods of research evaluation, at the macro- 
level and at the level of description. 

The distinction implied between 
intersubjective criteria and objective measures 
might seem technical but it does have 
significant implications. First, it enforces the 
recognition that citations are a product of 
human activity and not naturally given. 
Humans making citations may change their 
actions if the significance attached to citations 
is altered (as Laurie Taylor has more 
informally observed). Second, citation counts 
are no more, and no less, objective than any 
other collective human activity. 

At the level of value, we then move from 
using citation data as a substitute for other 
methods of research evaluation to a position 
where it could be used to inform judgements 
of research quality or as a forensic technique. 


77 


Aslib Proceedings: New Information Perspectives 
Volume 55 - Number 1/2 - 2003 - 75-83 





By restoring the distinction between 
description and value, and clarifying a 
contrast between macro- and micro- level 
correlations, the intellectual labour embodied 
in the studies demonstrating correlations is 
carried forward, with a potential application 
to more productive purposes. 


Data analysis and presentation 

The data relating to the public reception of 
the RAE was analysed. For the analysis of 
themes embodied in the public discussion, 
understandings and techniques derived from 
grounded research were used. Themes were 
identified and passages embodying themes 
were sorted under thematic categories. 
Categorisations are regarded as robust and 
complete when categories are relatively 
mutually exclusive, the overall categorisation 
is not too fragmented or complex, and when 
further data can be assimilated without 
change to categories (a distinction between 
the robustness and the analytical interest of a 
categorisation can be made, although it is 
often left implicit or assumed). 

The data set (only measured after analysis) 
amounted to c.400,000 words but was 
analysed without the use of specialised 
software packages. Direct immersion in data 
is valuable for the gradual accumulation of 
insights it can yield (compare native language 
acquisition with learning a foreign language). 
Not using a specialised package also enforces 
distinctions between theory (grounded 
research), technique (categorisation of data), 
and method (sorting data into categories), 
which might otherwise tend to become 
obscured. For more extensive data sets, 
greater human labour in analysis would be 
required, possibly assisted by specialised 
software packages. 

Results of analysis were presented verbally, 
in written form. For other studies, this could 
be supplemented by graphic presentation of 
dissemination over time and intellectual 
space. In contrast to more standard 
techniques, such as questionnaires and 
interviews, for assessing public or participant 
understanding of an event, data derived from 
public discussion can be more highly current, 
with less immediate human labour and 
associated cost. It is also less artificially 
elicited than interview or questionnaire 
responses (although the content of messages 
is still clearly influenced by the circumstances 
of their production). 
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The ethical responsibilities attached to the 
production and publication of citation data 
constitute a final issue relating to the 
empirical focus of the project. The issue 
emerged into awareness after the primary 
study and again reflects the case that 
conscious human subjects are both the 
objects and potential recipients of studies. 
What effect do published analyses have on 
research judgments? Do citation counts and 
research judgments remain independent 
variables after analyses have been published? 
An analogy with judicial procedures 
concerned with protecting jurors from bias 
would suggest that once studies are published 
they would affect judgment. What 
responsibilities should then attach to the 
publication and use of citation analyses? 

The issue of publication and 
communication brings us on to the 
communication model embodied in the 


paper. 


Communication model 


A communication model was developed 
concurrently with the acquisition and analysis 
of data. The model contains two crucial 
distinctions, between dissemination and 
diffusion and between esoteric and exoteric 
communities and media. Both of these 
distinctions have intellectual antecedents, 
which can be partially restored here. 


Dissemination and diffusion 
Dissemination is understood as message 
transmission and diffusion as the processes of 
understanding following on message 
transmission. 

A distinction between message transmission 
and understanding is deeply embedded in 
Western literate culture. For instance, it has 
analogues in the Platonic distinction between 
the sensible and the intelligible, in ordinary 
discourse references to form and content, and 
in semiotic distinctions between aspects of the 
sign. It can even be traced in information 
theory, where the communication engineer’s 
concern with a sequence of symbols is 
differentiated from a concern with the 
semantic aspects of the message. A distinction 
between message transmission and 
understanding may not be made by primarily 
oral cultures (consider the proverbial 
association of the messenger and message). 
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In this context, it should be noted that a 
contrast between message transmission and 
understanding has a biblical resonance, 
particularly in the parable of the sower, 
“Some fell upon stony places.” The parable 
distinguishes different depths and durations 
for diffusion and this could be a valuable 
discrimination. A further understanding 
developed is that insight can be crowded out 
by other concerns. An echo of the radical idea 
of dissemination, in the etymological sense of 
sowing seed, can be found in Bunyan’s verse 
preface to The Pilgrim’s Progress: “Then read 
my fancies, they will stick like burrs” 
(Bunyan, 1987b, p. 8). 

A distinction between dissemination and 
diffusion seemed essential, in the context of 
Warner (1997) and other analogous studies. 
A failure to make such a distinction could lead 
to conceptual confusion (compare not 
introducing a distinction of invention from 
innovation in assertions about the impact of 
technologies (Warner, 2000b)). Diffusion is 
regarded as more complex than dissemination 
and less amenable to central control and 
coordination. 

The crucial modern issue, debated in 
various disciplines, is whether a single plane 
of content is analytically adequate. A 
contrast could be made between Saussurean 
and Peircean semiotics, where a single 
signified is distinguished in the Saussurean 
tradition while Peirce (and Peirceans) make 
further analytical distinctions. For semiotics, 
the issue is whether the further complexity is 
repaid in terms of insight gained or 
systematicity enforced. By analogy, a single 
plane of diffusion might not be analytically 
adequate, particularly for accounting for 
different depths and durations, but further 
formalisation, as distinct from alertness to 
possible contrasts, might lead to rigidity. 


Esoteric and exoteric 

A further distinction was made between 
esoteric and exoteric communities and 
media (probably derived from an 
unconscious memory of structuralism). The 
distinction also has a theological resonance 
and origin, and now occurs primarily in 
theological discussions, but can be 
understood more broadly. In view of its 
admittedly ad hoc formulation, the scope, 
comprehensiveness, and possibility of 
effective recursive application of the 
distinction is surprising. 
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The distinction may be first grasped 
through exemplification. For instance, where 
information science is taken as the discipline 
for discussion, a series of disciplinary journals 
can be seen (further distinctions by 
communicative function could be made 
within these, signalled by the inclusion of a 
Bulletin as well as primary research journals) 
(Roberts, 1979). Scientometrics could be 
regarded as representing a sub-discipline. The 
distinction between esoteric and exoteric 
media can be applied recursively, to useful 
effect (see Figure 1). 

The next level is of trans-disciplinary 
publications, which have a crucial mediating 
function, communicating across disciplines, 
from scholarly research to the wider public, 
and from the wider public to scholarly 
communities. There are only a few, relatively 
high circulation publications in this area, but 
they have crucial mediating functions. Their 
low number could be regarded as an instance 
of the self-organising nature of 
communication, where market forces restrain 
the introduction of other journals. 

At the next level, we have educational 
supplements to national newspapers, which 
can be regarded as esoteric elements placed in 
an exoteric context, with the language and 
level of granularity of discussion adapted to 
that context and function. And, finally, we 
have the exoteric or wider public media 
themselves. 

There are dangers of introversion and 
isolation where an esoteric community 
becomes detached from the exoteric sphere 


Figure 1 Esoteric and exoteric media 
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(and, historically, this has been regarded as an 
aspect of the impoverishment and decline of 
the esoteric, from higher knowledge to the 
arcane). No a priori assumption need be 
made concerning the relative status (truth 
value) of esoteric and exoteric knowledge. 
Impoverishment of an esoteric community 
may occur if that esoteric sphere is isolated 
from exoteric discourse and the possibility of 
renewal by recruitment from the exoteric 
sphere. Some hints of such impoverishment 
could be found in the recent history of 
information science. 

The model of communication formalised 
here is more informally held by scientific 
participants in communication, revealed, for 
instance, in the significance attached to 
exoteric communication and to the journals 
with a mediating function (consider the 
prestige of publication in Nature, for 
instance). Contributors to particular debates 
will act upon the model informally held. In 
this sense, like other actors in a labour 
process, scientists and other scholars both 
produce the system of communication and 
are confronted by it in the appearance of 
objective reality. 


Intellectual space 

An idea of intellectual space is implicit in the 
model for communication. The idea of 
intellectual, as distinct from physical or 
geographic, space is partly metaphorical but 
so widely used that its metaphorical elements 
may be forgotten. In their radical or 
etymological senses, esoteric and exoteric 
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refer to place, to within and outside. In 
primarily oral societies, utterances were 
inherently located in space and time and were 
not just metaphorically placed. An idea of 
holy or sacred space, in which only certain 
utterances are permitted, still persists. 


Mediating function 

The mediating function taken by certain 
publications need not be bi-directional. While, 
for example, Nature both communicates with a 
number of disciplines (also functioning as a 
trade or professional journal) and, partly 
through press releases, with the exoteric 
public, other trade journals of academia, such 
as the Times Higher, communicate primarily 
with the scholarly sphere. They may be used 
by policy makers in the exoteric or wider 
public sphere to communicate with the 
scholarly sphere as a whole. 


Proximity of the utterance within the 
discourse community 

An idea implicit in these discussions, 
particularly in the directionality of the 
mediating function, is that of the proximity of 
the utterance within the discourse 
community. Proximity is not necessarily 
bilaterally equivalent in strength: for instance, 
the readers and authors of Scientometrics will 
regard utterances about bibliometrics in 
exoteric spheres as highly proximate while 
utterances in Scientometrics are generally 
ignored by the wider public sphere. 

The power of the idea of the proximity of 
the utterance within the discourse community 
can be realised when it is understood also to 
constitute the test of trademark infringement, 
applying across changed geographic extent for 
electronic commerce. It could also be 
detected in widely held, although not 
necessarily formalised, conventions for 
seminars, where, if a member of the audience 
is known to be strongly identified with a view 
opposed to that of the speaker, registering of 
disagreement is expected. 


Infrequent convergence of esoteric and 
exoteric spheres 

Studies of the dissemination of other scientific 
(more natural scientific) topics conducted by 
science communication students (who had 
indicated that they found the way of thinking 
embodied in the original paper very helpful) 
supported the suspicion that there is relatively 
infrequent convergence between the esoteric 
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and exoteric spheres. Convergence could be 
identified as specific communicative events. 
Between events of convergence, the esoteric 
and exoteric spheres tend to function 
autonomously, until further instances of 
connection. In terms of the communication 
model, this is supported by the relatively small 
number of mediating publications, compared 
to the diversity of esoteric and exoteric 
spheres. For the purposes of data analysis for 
more extensively reported topics, isolating 
such events would enable a meaningful 
restriction on the extent of data to be 
analysed, with a focus on crucial transitions. 


Disaggregated knowledge 
One generic theme which emerged from the 
Warner (1997) study was the possibility of 
disaggregated public knowledge, of 
understandings which could be derived and 
synthesized from a number of contributions, 
but which are not present in any single 
contribution. In relation to the RAE, the 
influence of place on potential for research 
was detected and the strongly patterned 
distribution of research excellence within the 
United Kingdom has been noticed by other 
studies (including Lewison, 2003). Curiously, 
in the public discussion of the RAE 1996, this 
was implied in the repeated analogies made 
between the league tables of universities and 
those of professional football (some will recall 
the correlation between the founding 
members of the Football League and the 
locations for the industrial revolution) and 
was not made fully explicit. Academics, as 
social beings, may be attracted to pleasant 
locations and there are reinforcing effects. 
Students critiquing papers suggested that 
such insights should be taken outside the 
communication model and studied 
separately. This implies that the 
communication model, while robust and 
analytically revealing with regard to forms of 
communication, is not necessarily analytically 
revealing with regard to topics discovered. 


Predictive value of the communication 
model 

The distinctions embodied in the 
communication model proved capable of 
describing and predicting literature about the 
RAE. A level of granularity of literature 
indicated in the original paper, but not 
directly investigated, was the intra- 
disciplinary discussions, which tended to 
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focus on the discipline of assessment. One 
contribution not immediately assimilable to 
this model was Warner (1997) itself, which 
was not concerned with the discipline of 
assessment or with using bibliometric 
techniques. It could be regarded as esoteric 
contribution with the possibility of wider 
exoteric interest. 


Information retrieval techniques 


Information retrieval techniques and 
resources were the basis for the acquisition of 
the data for the paper. In contrast, to paper 
indexes or serial searches of media, far less 
labour is required at the point of searching to 
identify and retrieve data. Technological 
developments have made extensive social 
epistemological studies possible. 

Information resources are developing and 
patterns of provision are changing. To 
conduct an analogous study of the RAE 2001 
would be more complex: there has been some 
migration of files of newspapers to other 
providers and, more significantly, a growth of 
institutional Web sites, creating problems of 
selection and filtering of a mass of material. 

Recall of literature on the Research 
Assessment Exercise was possible through 
phrase searching (good editorial practice leads 
to full mentions before acronyms are 
introduced). With the further diffusion of the 
topic, acronyms and indirect discussions may 
have become more common, creating 
problems for full recall (RAE also exists as a 
name). Other topics, for instance copyright in 
computer programs as literary works, would 
present severer — effectively insuperable — 
problems of isolation and recall of relevant 
documents, given the multivalence of 
potential query terms when abstracted from 
their discursive context. Direct scanning of 
media at selected dates (for instance, 
following the parliamentary discussion or 
instantiation of copyright legislation) would 
be, and has been, required. 

Information resources and techniques are 
well understood within information science, 
but less well known and understood in other 
disciplines, although knowledge of their 
existence has been increasingly diffused 
through Internet search engines and 
directories. There would be potential for 
export of understandings and techniques 
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from information science to other esoteric 
spheres and to the wider public. 


Information transfer: an explicable 
process 


One value of the approach of Warner (1997) 
is that it demonstrates, and does not simply 
assert, that information transfer is an 
explicable not a mysterious process. The 
approach could be described as materialist, in 
that the transference of an idea is never 
assumed without looking at the particular 
mechanism of transmission (and the 
methodological exclusion of oral 
communication is not confused with any lack 
of substantive significance). 

Science communication students were able 
to apply techniques and the communication 
model to other, more natural scientific topics, 
demonstrating the robustness and utility of 
the methods for retrieval and analysis of data 
and of the distinctions made in the 
communication model. The esoteric-exoteric 
distinction could be replied recursively and 
the esoteric need not be solely understood as 
the scholarly. Discussion at the conference 
revealed analogies between the processes of 
dissemination of scientific information (by 
press releases connected with conference 
papers) and government statistical 
information (press releases in connection with 
major publications, such as Social Trends). 
Recent historical changes have not directly 
disturbed these patterns but have enhanced 
possibilities for direct public viewing of 
primary sources, by electronic access. 

Students also used citation analysis for 
tracing the diffusion of ideas in esoteric 
scholarly spheres (their concern was not 
immediately with comparative evaluation of 
the quality of published research). We now 
have techniques and a historically 
unprecedented opportunity for recovering 
public knowledge in the scholarly and wider 
public sphere and studying it, at various levels 
of aggregation. 


Further development 


What further developments have been 
considered? One strategy would be to 
contextualise and develop distinctions, 
drawing on the philosophy of science, to 
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recover and substantiate distinctions made 
between esoteric and exoteric circles in 
science, particularly by Ludwig Fleck (1979). 
The sociology of texts, developing from the 
1985 Panizzi lectures and becoming 
increasingly influential would be a further 
resource (McKenzie, 1986). 

Such contexts could be both legitimating 
(identifying social epistemological studies as a 
legitimate domain for information science) 
and informative (developing distinctions 
made). To date, they have been more 
legitimating than informative, except in so far 
as encouraging a restriction on further model 
elaboration. At one stage further elaboration 
of a rather skeletal communication model was 
considered. Yet dialogue with students 
suggests retention of the basic distinctions in 
the communications model and a 
concentration on greater descriptive richness, 
obtained through a series of studies, rather 
than direct model elaboration: 


Prior knowledge of the topic coupled with 
intuition as to the result, lends a significant 
subjective quality to the study. This does not 
detract from the model and techniques used, it 
does instead realise the potential that such a 
simple model encapsulates (Comments received 
from MSc Science Communication student, 
2001). 


Model elaboration might reduce the 
simplicity and attractiveness of the 
distinctions made. Even Fleck, although 
carefully distinguishing esoteric and exoteric 
circles in science (there are both analogies and 
contrasts with Kuhn’s work), does not 
formalise these distinctions further. 

Patterns, both of communication and 
information resources, are changing. In an 
irony not unknown in system modification, 
the forces (the intensification of electronic 
communication) which make such studies 
possible are calling the categories established 
into question. Locating Internet 
communications in intellectual space is more 
difficult than for printed journals. There is 
some evidence that communication with 
other disciplines is facilitated by publication 
in electronic journals. Electronic publications 
can therefore be regarded as closer to 
transdisciplinary space than printed 
documents, whose circulation may be more 
restricted, without additional document 
retrieval labour by the reader. Previously 
separate esoteric spheres may be converging 
as part of this process. 
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Non-written public knowledge exists, 
including oral and graphic forms, although 
these were excluded on methodological 
grounds from Warner (1997) and student 
studies. Forms of communication other than 
written language, including both oral and 
non-verbal graphic communication, may 
require greater human labour in indexing and 
the costs of human labour may discourage 
production. In addition, legal deposit and 
library collection practices have been oriented 
towards printed not broadcast media. The 
omission of non-written public media 
inevitably restricts the scope of studies. 

A unique historical opportunity for 
transforming the social epistemology 
envisaged by Shera and Egan in the 1950s 
(Shera, 1965) into recurrent research practice 
exists. Shera and Egan’s primary intention for 
social epistemology was to inform 
bibliographic arrangements, by matching 
patterns of bibliographic provision to patterns 
of intellectual consumption. The concern 
here is with the distribution of public 
knowledge and possibility of synthesising 
disaggregated public knowledge. The 
opportunity for social epistemology could be 
regarded as the Word made flesh. 

There are emerging signs of cognate 
interests, independently developed, in 
economics. A crucial issue in the developing 
field of information economics will be, “how 
and how well organizations and societies 
absorb new information, learn, adapt their 
behavior, and even their structures; and how 
different economic and organizational designs 
affect the ability to create, transmit, absorb, 
and use knowledge and information” (Stiglitz, 
2000, p. 1471). Descriptive richness in social 
epistemology could then be assisted by the 
collaborative labour of other disciplines. 
Information science could contribute 
information retrieval techniques and an 
understanding of communication, 
particularly scholarly communication. 


Conclusion 


The freedom of academics to reflect and write 
is enabled by forms of labour (agricultural, 
industrial, and commercial) which are 
immediately essential to communal survival 
or at least command a more direct exchange- 
value. There would, then, be an implied 
responsibility or virtual contract to return the 
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products of the intellectual activity enabled to 
exoteric labour, for instance, to inform policy 
(which might also have indirect rewards in 
support for research). Roland Barthes, quoted 
in Warner (1997), insists that the task of 
research must be perceived in desire and that 
desire arises from reader demand not direct 
institutional payment. One aspect of reader 
demand might be exoteric communication. 
Some related issues emerging from 
discussion with conference participants were 
that attitudes which form grounding 
assumptions for academics and whose 
strength is matched by the infrequency with 
which they are articulated may not be shared 
by policy makers, including elected 
politicians, with whom they attempt to 
communicate. Rationality, using evidence to 
support assertions, valuing formal written 
communication regarded as an end in itself 
(reflected in the prestige attached to 
publication) are fundamental to academics’ 
professional ethos. Policy makers, by contrast, 
may prefer oral communication (with a 
strategic balance between monologue and 
dialogue), may value formal written 
communication as a means of action rather 
than as an end in itself (consider legislation as 
action), have weaker notions of rationality, 
and be guided by strategic considerations, 
including electoral advantage, in their choice 
of policies to support. To communicate with 
policy makers, adjustment must be made to 
their framework of persuasion. If academics 
cling too rigidly to their grounding 
assumptions (and the first step in liberation 
might be to recognise them) they may lose the 
opportunity of influence and invite the 
response Faithful received from Bunyan’s 
jury, “Hanging is too good for him,’ said 
Mr Cruelty” (Bunyan, 1987b, pp. 85-6). 


Note 


1 Students were taking a MSc Science 
Communication run jointly between Queen's 
University Belfast and Dublin City University, in 
1997-1998 and 2001-2002. The 1997 cohort were 
given the opportunity to compare the approaches of 
Warner (1997) and Oppenheim (1995), as an 
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alternative amongst information resources 
assignments. In a more developed programme, the 
2001 cohort were asked to carry out studies of the 
dissemination of scientific topics in scholarly and 
wider public domains, using citation analysis and 
the communication mode} and techniques of 
Warner (1997). Reflection on the value of the 
techniques and the communication model, in 
relation to the specific topic studied, was also 
requested. 
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Abstract 


Evaluates, through deep log analysis, the impact of 

“Big Deal” agreements on the online searching behaviour 
of users of the Emerald digital library Web site, which 
provides access to more than 150 journals in the fields cf 
business and information science. The purpose of the 
evaluation was to map the online information seeking 
behaviour of the digital library user and to see whether 
those signed-up to a Big Deal arrangement behaved any 
differently from the others. In general they did. The real 
surprise proved to be the strong consumer traits of the 
library's users. Research reported here refers to the first 
stage of a three-stage research project. 
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Introduction 


The “Big Deal” was introduced as a solution to 

the “serials crisis” and was first developed 

during 1997 by the leading Library 

Consortium OhioLink and Academic Press, 

and then taken up by others including Elsevier 

Science, Blackwell and Emerald. The basic 

essentials of the Deal are that the publisher 

continues to receive the same amount of money 

as they received from print subscriptions from 

all the libraries in a consortium. For a small 

surcharge, all the libraries in the consortium 

receive electronic access to all the journals 

published by a particular company. 

Tom Sanville (2002), the prime begetter of this 

idea lists the advantages of the Deal as: 

e Increasing content delivered per dollar/ 
pound spent. 

+  Stabilising economic relationships. 

+ Maximising content delivery per 
publisher. 

* Maximising use via electronic access. 


The Big Deal has triumphed in 

North America. Only two serious research 
libraries have not taken it up. In the UK the 
reception has been somewhat mixed. 

There are a number of important 
information players in the digital journal field 
with strong vested and entrenched interests, 
and they all believe — or state — that they are 
serving the interests of the reader or consumer 
— who are mainly, but by no means exclusively, 
academics. The Deal is in fact described as a 
win/win/win situation in that all the 
participants, libraries, publishers and authors/ 
users benefit from it. However, the point about 
the Big Deal is that it is a deal jointly decided 
by and jointly backed up by both libraries and 
publishers and perhaps authors also, for whom 
the advantage to the authors is wider access to 
their material. But what are the real and 
perceived advantages to the user? Indeed, 
generally, what do the users do? This is what 
this paper attempts to seek. It is about the 
actions of the users rather than their views. 
The bigger project will cover views. 

Despite widespread industry interest in what 
is plainly a strategic topic to all interest groups, 
large scale and comprehensive evaluations of 
the changing use patterns that are manifesting 
themselves as part of widespread electronic 
access to journals, through such programmes 
as the Big Deal, are surprisingly thin on the 
ground. There are a number of studies devoted 
to working out how to obtain and how to 
handle usage statistics (see for instance, 
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Montgomery, 2000; Mercer, 2000), but 
analyses concerned with evaluating the Big 
Deal in particular have largely come from 
OhioLink — and their findings are thought to 
be controversial, selective and tainted by self- 
interest. Matters are further clouded by the 
fact that, due to data protection 
considerations, consumer confidentiality and 
commercial considerations, very little data or 
evidence has in fact been released. 

Logic and our experience from other fields 
— the media and health[1], would lead us to 
believe that something major has probably 
occurred as a result of the user: 

* being provided with easy access to data 
that was previously far from easy to access 
(from both a physical and information 
retrieval point of view); 

* being provided with massive journal and 
article choice, which might lead to the 
kinds of “promiscuous” behaviour you 
see on the Web; and 

* offered a level playing field; all members 
of the Deal — big, small, urban and rural, 
have the same levels of access; arguably 
small organisations profit most. 


It seems likely ~ although clearly we should 
not rush to pre-judge things — that improved 
access and choice should be changing things; 
whether that is slowly, quickly or patchily we 
are not quite so sure. We are already getting 
wind of what these changes might be. Thus 
data from OhioLink appears to show that 
some of the most used journals are not those 
that the libraries originally subscribed to in 
print form. If true it shows the development 
of the classic consumer trait of people 
shopping around, making up their own 
minds. It may also put a question mark over 
journal selection policies ~ and the role of the 
intermediary, too. Or, as Anthony Watkinson 
(2002) has argued, it could be down to the 
new types of people who are accessing the 
journals. The reason for the impressive usage 
electronically of journals not selected for 
purchase in print could be owing to the non- 
discriminatory activities of students writing 
term papers or preparing for research. 
Interestingly, an article by Phil Davis (2002) 
based on NERL data comes up with 
contradictory results to OhioLink. Plainly, we 
need to investigate this further for the truth. 
Further data from OhioLink appears to 
show that journals are not substitutable for one 
another (Kohl, 2002). In other words, as new 
journals are added to the database, there 
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appears to be no decline or slowdown in the 
use of journals of previously loaded publishers. 
The assumptions being that patrons are not 
switching from one title to the other. In other 
words, there appears to be no signs of 
consumer promiscuity. For the sciences this is 
thought to be axiomatic that each journal is a 
unique collection of papers with characteristics 
in common (Mabe, 2002). You might expect 
more porosity in the humanities and social 
sciences. Overall, it is thought that the STM 
element predominates simply through sheer 
numbers and therefore the STM effect 
predominates for journals as a whole. 

Finally, Don King (2002) has found that 
increased access to electronic journals may 
result in scientists reading from a broader 
range of journals and spending less time on 
each individual journal. Certainly this is our 
experience of the Web. 

The authors were commissioned by the 
Ingenta Institute specifically to test the above 
data and assertions, and more generally to 
determine what impact Big Deals were having 
on the user. 


Aims and objectives 


Big Deals for the user are essentially all about 
choice and enhanced access and the project’s[2] 
broad brief was to determine whether this has 
resulted in any changes or differences in 
behaviour, both online and offline. The specific 
project aims were as follows: 

(1) To chart through usage logs how the user 
behaves in the digital information domain 
of plentiful choice created by the Big Deal 
and to identify any changes that were 
occurring in information seeking behaviour. 

(2) On the basis of this data, set about 
framing interview questions and 
questionnaires for Big Deal (and non-Big 
Deal) users about their information 
seeking in the digital environment. 

(3) To determine what all this means in the 
wider academic, economic, political and 
social context. In other words what 
outcomes result from the widespread 
availability of digital journals occasioned by 
the Big Deal — better research, improved 
teaching, better control of events? 

(4) Examine the logs in the light of the 
experience we have obtained from our 
deep log evaluations of the newspaper 
and health fields. 
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This paper provides early sights, an early 
“cut” of the data that are emerging, which 
largely relate to the first aim named above. 


Methods 


Case study approach 

There was a need to pilot and hone our 
methodology and in this connection the Big 
Deals of two publishers were chosen for 
investigation. British publishers, albeit with 
international portfolios, were chosen because 
of convenience and the fact that they were 
more likely to release data to us (US 
publishers and librarians are more wary of the 
data protection aspects of log data). We 
needed publishers who had much to learn 
from the research (and therefore would be co- 
operative) and had a close grip on the 
situation so we could drill-down and obtain 
causal data and general explanation from 
publisher, client organisation and user. 

The two publishers that fitted the bill were 
Emerald and Blackwell Science. The fact that 
‘one was a business/social science publisher 
and the other a STM publisher, one a 
medium sized publisher and the other a large 
publisher was an added bonus. There are 
undoubtedly problems in making 
comparisons between vendors. Indeed, most 
library interests (such as ARL) do not believe 
this type of comparison will be possible until 
standards such as COUNTER (http:// 
www.projectcounter.org/news.html) is 
working on become generally accepted. We 
have some experience of making comparisons 
between the logs of different digital platforms 
(Nicholas et al., 2001a). 

Presented here is work in progress 
concerning Emerald only. 


Logs 
As mentioned above log analysis was the 
methodology of choice. All digital 
information services, including digital 
libraries like Emerald’s, have a facility by 
which logs are generated that provides an 
automatic and real-time record of use by 
everyone who accesses the system. The kind 
and amount of data that can be harvested 
varies between platforms and services but 
essentially the data is much the same. Log 
analysis is all about squeezing the maximum 
data from the skeletal logs and carefully 
interpreting the resulting information to 
provide those important initial sketches of the 
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information landscape, which provide the 
foundation for the long-term analysis of 
change and the source of questions for use in 
questionnaires and interviews. 

There were a number of important reasons 
why we majored on log analysis. First, logs 
provide fairly concrete and abundant evidence 
of use — and also somewhat less concrete 
evidence of satisfaction. With log analysis it is 
possible for researchers to monitor the use of a 
system by millions of people around the world. 
This is a far cry from some of the assessment 
and monitoring methods of the past, which 
were based on small and unrepresentative 
samples (a few dozen “tame and pliant” users — 
students and academics typically). Logs record 
use by everyone who happened to engage with 
the system, there is no need to take a sample. 
This might, of course, lead us to question the 
validity of what we have learnt from past 
studies employing less robust qualitative 
methods. The great advantages of the digital 
logs are not simply their size and reach, 
although the dividend here is indeed a rich and 
unparalleled one. Just as important is the fact 
that they are a direct and immediately available 
record of what (thousands/millions of) people 
have done: not what they say they might, or 
would, do; not what they were prompted to 
say; not what they thought they did (the 
traditional domain of questionnaires). The data 
are unfiltered and speak for themselves. This is 
especially important in an area, like digital 
information use, where users are easily 
intimidated and new to the whole thing. There 
is nothing wrong with questionnaires and 
interviews but used on their own — and in a 
vacuum — in a rapidly emerging, fluid, difficult 
and sensitive area, they may prove misleading. 
Yet used to investigate further the patterns and 
issues emerging from the logs and they then 
become energised and powerful research 
instruments, which is how we shall use them in 
the second part of the study. 

Second, because the data are being 
collected routinely, seamlessly and 
continuously it means changes can be 
mapped as they occur. Something that has to 
be very important as it is still early days 
systems-wise and many consumers are yet to 
fully engage with the world of digital 
information consumption and make their 
presence felt. Dynamic methods are needed 
to study fast changing environments — and 
this might be the case in this particular 
environment[3]. Of course, we are only 
offering up a snapshot in this particular paper. 
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Server logs were obtained for the whole of 
June 2002. The entire log file consisted of 4.5 
million lines spread over about 20 servers — a 
significant dataset by any count. The file 
arrived with log information combined from 
the different servers. The log files are not 
typical of Internet log files in that the logs do 
not record files delivered but requests for 
document forms. An example of Emerald log 
lines is shown in Table I. 

Lindoro 1791 refers to the server; what then 
follows is the date and time of the request. 
Following on in block capitals is the type of 
request. WWWISSUES is a request for a 
table of contents for a journal issue. After that 
we get a whole range of lower case alphabetic 
codes. They have the following meanings: 

* Ac = account: the account (CID) that 
provided access to the content. 

+ Acs = temporary account: the accounts 
that the user was recognised as. 

* User = the username entered into the 
login form. 

* Pas = the password entered into the login 
form. 

* Nw = the network that the content is held 
on — “1” for all Emerald Content. 

* Ung = the unique identification number 
used by the server to write and read user 
cookies. 

* Val = the equivalent of the VL number on 
the URL - the validation number is 
internal reserve information used to 
identify and track the user. 

* Cns = the contracts which the user has 
been recognised as having in the same 
format as the “acs” item. 

e Pubs = the number of and identification 
of any publishers the user has been 
recognised as. 


Table | Example of Emerald log lines 
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« Acsnew = the number of new accounts 
the login action has added to the user 
information. 

* Cnsnew = the number of new contracts 
the login action has added to the user 
information. 

* tt = the title identifier or titnum internally 
used to identify a journal. 

* Pb = the publisher identifier used 
internally to identify a publisher. 

* Vi -the volume number of the requested 
item. 

* Nm -the issue number of the requested 
item. 

+ Sp —the supplement identifier of the 
requested item. 

* Sc -— the section number of the requested 
item. 


Results 


Users and categories of user 

The population of users was determined by 
those who used the Emerald site. This means 
that the population is not purely academic in 
nature. Thus we will also have picked-up 
academics, students, researchers and 
practitioners. In fact, an analysis of IP 
addresses shows that just 45 per cent of users 
came from an academic institution. 

The Emerald site might be best thought of 
as a large digital departmental store, which 
provides customers with plenty of opportunity 
to look around and sample the products and 
services on offer, and then provides a range of 
payment methods by which they can take the 
full-text product away with them. All 
products have a high degree of digital 
visibility. Access to the Emerald site is then 


lindoro: 1791 24/6/2002 11:23:24.51 13: WWWISSUES acs=11014 tt=1148 pb=106 vl=0 nm=0 sp=0 sc=0 sc=0 


ung=63498784 ver=5000 inizemerald bini= wis=emerald 


lindoro: 1791 24/6/2002 11:23:29.99 18: LOGIN addr=65.214.36.158 user= pass= nw=1 vers=5000 
ung=1415940992 port=WebTcp val=776911 ini=emerald bini= wis=emerald cns=0 acs=0 pubs=0 acsnew=0 


cnsnew=0 brows="AskJeeves” 


lindoro: 1791 24/6/2002 11:23:29.99 18: NONFREE tt=1152 pb=106 vl=37 nm=1 sp=0 sc=1 unq=1415940992 


ver=5000 ini=emerald bini= wis=emerald 


lindoro: 1791 24/6/2002 11:23:30.61 18: ABSTRACT tt=1152 pb=106 vl=37 nm=1 sp=0 sc=1 ung=1415940992 


ver=5000 ini=emerald bin= wis=emerald 


lindoro: 1791 24/6/2002 11:23:39.65 19: LOGIN addr=65.214.36.158 user= pass= nw=1 vers=5000 
ung=1206048384 port=WebTcp val=1000494 ini=emerald bini= wis=emerald cns=0 acs=0 pubs=0 acsnew=0 


cnsnew=0 brows="AskJeeves” 


lindoro: 1791 24/6/2002 11:23:41.30 13: WWWCONTENTS acs=11014 tt=1148 pb=106 vl=7 nm=2 sp=0 sc=0 


ung=63498784 ver=5000 ini=emerald bini= wis=emerald 
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open to everyone with Internet access, but 

full-text download rights vary considerably 

according to the class to which the consumer 
belongs, with Big Deal consumers having the 
best choice and access. A cookie reference 
and a user id number annotate all lines in the 
logs, which means we can identify the class, 
and most importantly the Big Deal 
subscriber. Thus by comparing this data with 

Emerald’s list of subscribers it is possible to 

determine who were subscribers and who 

were not, and of the subscribers, who were 
signed-up to the Big Deal and who were not. 

Because our analysis was based on the 

“cookie reference” even if the user looked at 

something without entering their user id 

number we would still pick up the access as a 

subscriber access — we linked cookie to user 

id. In fact, the logs record a number of user 

identification codes: the subscription code 

that the user has logged in with, the user’s 
login name and password, the IP number of 
the requesting client, as well as the cookie 

` identification number. 

Thus from the logs it proved possible to 
distinguish the following categories of user, 
which enabled us to benchmark the activities 
of the Big Deal user: 

* Non-subscribers. These people can browse 
or search for journal articles, view tables of 
contents and abstracts and sign-up for an 
alerting service without supplying a 
username and password. Emerald is doing 

‘more than just laying bait — hoping that 
non-subscribers will convert to subscribers 
by allowing users so much freedom. For 
these people can also download full-text 
articles but, unless they were in Journal of 
the Week titles (see later for details), for 
which downloads were free, they will need 
their credit card to do so, which as we shall 
learn later they do use. Non-subscribers, of 
course, must form a very heterogeneous 
group. They could be anyone, but are 
most likely to be: students, using the 
system as an index to library collections 
from home; foreign users who were glad 
enough to have abstracts of UK material; 
the general digital information consumer 
just surfing around, coming in from a 
search engine; and robots (as there was no 
robot.txt file present, robots could not be 
identified before the analysis, hence robots 
are included in the analysis of non- 
subscribers). 

Another group of users included in the 
non-subscriber category was people 
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opting for a free trial subscription. They 
are limited to viewing five articles in full- 
text — thereafter they have to buy (e.g. 
pay-per-view), have a 30-day trial period 
and are requested to fill in a registration 
form. We have been able to isolate this 
group; we call them trialists, and some of 
the following analyses feature this group. 
It should be noted that Emerald is 
particularly keen on offering trials and the 
sorts of trials they offer are unusually 
generous by publishing standards. 

Non-subscribers provide an interesting 
group against which to benchmark the 
behaviour of subscribers, who are more 
likely to be the traditional academic 
library user. 

* Subscribers. They may have multiple login 
codes. There are two types. Non-Big Deal 
subscribers have limited access to a range 
of journals depending on the nature of the 
subscription. Big Deal subscribers, there 
were about 900 of them, get full-text 
access to 118 out of 154 of Emerald’s 
journals. The vast majority of Emerald Big 
Deals are made through consortia. Within 
Emerald’s Big Deals, no organisation’s 
original subscription involved more than 
60 journals. In fact most of them 
subscribed to less than ten journals. What 
basically happened was this. An 
organisation subscribed to four journals, 
they negotiated,a Big Deal (for three 
years), then were provided access to 118 
journals plus their four journals (which 
may or may not have been part of the 
Deal) and must maintain their spend for 
the three years of their agreement. What 
this all meant was that the vast majority of 
subscribers significantly increased the 
range of journals they had access to. 


Not all Emerald journals are in the Big Deal 
because some journals were acquired after the 
Big Deals had been signed and also some 
subject areas (e.g. engineering) are not 
included as part of a Big Deal. 

During June 2002, 154,536 “people” used 
the service, a testament to how popular digital 
libraries, like Emerald’s truly are. Of these an 
astonishing 96,908 (63 per cent) were non- 
subscribers, including trialists, who 
accounted for 7 per cent of all users (Figure 
1). Plainly this is an open consumer site in 
anyone’s terms, and has little in common with 
its predecessors like Dialog and DataStar. It 
also suggests that there are plenty of casual 
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Figure 1 Percentage of users by type of user 
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users seeking journal articles, certainly in the 
social sciences anyway. There were 54,741 
Big Deal asers and 2,887 non-Deal users. 
The Big Deal is by far the most popular 
subscriber option, and is a further testament 
to its success. Of subscribers, 2 per cent of all 
users were non-Deal subscribers and 35 per 
cent of users were Deal subscriber users. 

Figure 2 shows that number of users varied 
quite considerably from day to day, from 
4,000 to 9,000 users. Approximately 208 
people us2d the system per hour. 

Of course, any comparison between user 
groups is complicated (and limited) by the fact 
that we have.no way of knowing whether non- 
subscribers or non-Big Deal subscribers were 
a party to other publishers’ Big Deals. 
However, qualitative work will help us to 
establish this. There is another problem too. 
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The cookie on a computer tracks use by that 
person on that machine. However, some 
computers, like those in the library, are shared 
and we have no way of knowing this. 
Unfortunately, we cannot get much closer to 
users than this at present. This is because the 
vast majority of Big Deal subscribers employ 
IP authentication. IP addresses are either a 
central firewall IP or dynamically allocated at 
the time of use. Both of these methods do not 
lend themselves to identification of user class — 
department for instance, which would have 
been very interesting. The Emerald subscriber 
list does not furnish these details either. It 
would be good to determine the user’s age and 
gender too, but that data will have to be 
obtained by other research methods. It would 
be good to identify use by librarians too. Their 
use is likely to inflate the use figures — possibly 


_ quite substantially. A cookie audit at a case 


study institution might fill in our knowledge 
gaps — we are currently investigating this. 


Use 


Defining use : 

If the term user is problematical when used in 
log studies, then what constitutes use.and 
how you can measure it is far more so (for a 
discussion see Pullinger and Baldwin, 2002). 
Indeed, the word use itself has been 
questioned — it might be argued that use is not 
being directly measured. It might be more 
accurate to claim that what is being measured 
are “accesses”, downloads, etc. We prefer the 
term “page requests” but do not have a 
problem with use. 

Activity is clearly the defining characteristic 
of use, but what actually signifies it and 
whether all forms should be treated equally 
are the issues that need resolving. There is 
much potential for confusion. This arises 
partly from the use of the term hits, which is ` 
widely used by Big Deal vendors (like 
Emerald) and general Web sites alike. 
However, they are often using the term 
differently. There is an Internet convention to 
resolve use down to pages viewed — and this is 
in fact how we get the term hits[4]. In regard 
to a typical Web site, hits encompass all kinds 
of activity — arrivals by accident, people 
navigating through the site, viewing content, 
downloading a page, using a page as office 
“wallpaper”, whatever. On the Web all hits 
(pages) are counted. In the case of Big Deal 
vendors the word hits is used in a rather more 
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restricted way. It is most often used to mean a 
request — and a request for a “useful” page at 
that — like an abstract, full-text article etc. The 
most selective use metric employed is 
“downloads”, and this refers to articles 
viewed or downloaded in full-text. Plainly this 
metric shows real interest and motivation, and 
from Emerald’s perspective it is also a revenue 
generator, albeit a very small (but growing) 
one at present. Of course, the really big 
revenue generator for Emerald is 
subscriptions and, once you have a 
subscription payment, you do not mind very 
much how much use is made. However, 
article usage will become a major 
consideration in Big Deal renewal discussions 
in the future — at both the journal and 
increasingly database level. As an indicator for 
the future, then, it is pretty important. 

Let us take the specific case of Emerald’s 
raw logs to illustrate the problem. They 
identify the following activities (in brackets is 
the reference in the logs — see Table I): 
Request for abstract page (Abstract). 
Request for article (Article). 

Request via internal site link (i.e. 
reference and other links) (CGILink). 
Request for article format selection page 
(Fulltextreq). 

Request for article for which they do not 
have a subscription (Nonfree). 

Request for table of contents for a journal 
issue (WWWissues). 


Request for contents page of all issues of a 


journal (WWWcecontent). 


Most are self-explanatory, but two are not, 
and can — and do — lead to some confusion. 
First, the request for an article format 
selection page (Fulltextreq). This in fact logs 
the request for the full-text of an article, not 
its download. When the request is made there 
is a prompt to enter username and password 
and once the username and password are 
verified the article request is fulfilled, so 
successful logins would be picked up by the 
request for an article metric. Of course people 
request the full-text for journal articles they 
are not entitled to, and do not want to pay for. 
The value of this metric is possibly 
questionable, but it does reflect activity. 
Second, there is the request for an article for 
which no subscription is held (nonfree). This 
logs cases where a user, who has logged on 
but does not have a subscription to a journal/ 
article, still tries to access the article. Instead 
they see the abstract page only. When such an 
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access is made the “nonfree” request is 
delivered with the “abstract” request and 
provides information that is added to the 
abstract request. Table I gives an example and 
shows that the “nonfree” and “abstract” 
request are delivered at the same time — in this 
case within half a second of each other. 

In fact, the hit statistics Emerald provide in 
their log reports for clients concern just the 
following metrics: 

Content page hits (content page hits 
include both types of content display — 
wwweontent and wwwissue). 

Abstract hits (an abstract hit is where the 
user sees the abstract page and the option 
to download on the same page). 

Article hits (downloads). 
Non-subscribed hits (confusingly 
described, as it is in fact, our “old friend” 
the nonfree abstract). A non-subscribed 
hit is where a user, who has logged on but 
does not have a subscription to a journal/ 
article, still tries to access the article. 
They see the abstract page only. 


We can see what Emerald (in fact, 
Catchword, the suppliers of the log reports) 
have done: 

They have identified what they believe to 
be useful behaviour and counted that. 
This explains why fulltextreq is not 
included. CGlIlink (which includes the 
citation search request) has been 
excluded on grounds of the inability to 
easily distinguish citation requests from 
other internal requests. 

They have separated out abstract counts 
according to whether the request was in 
fact for an abstract in the first place or 
whether the abstract was viewed as a 
result of an “illegal” request. Whether 
this separate counting of abstracts is 
warranted is open to doubt. 


A typical Internet site owner would include 
both fulltextreq and CGlIlink pages as “hits” 
and would not distinguish between the kind 
of hits. 

Of course, requests do not necessarily 
equate with pages or screens viewed. Thus 
when you do a search and view that search 
you might have to scroll ten screens to see 
what you want; now that is one request but 
many screens viewed. Similarly, for articles, 
one request retrieves an article but when you 
view it online it could span 30 screens, maybe. 

We are still experimenting with different 
use indices and we trial two in this paper — 
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“useful requests” — requests for articles, 
abstracts, and content pages and article 
requests only downloads (the most useful of 
all requests). We are also working on 
employing the reference link as a metric. 

It does not end there for once we have dealt 
with the problem of what to count we are 
confronted with the problem how to measure 
it. After all, there are many ways of measuring 
use and each way gives us a different “take” 
on how the system is used. They range from 
the simple numerical hit count to the more 
powerful metric of site penetration. 
Additionally there are time-based metrics. We 
are currently investigating the robustness of a 
number of use metrics and will in the future 
examine them in the light of COUNTER 
http://www. projectcounter.org/news.html, the 
industry’s log standard gurus, thinking. 


Actual use 

During June an astonishing 2,437,811 requests 
of all kinds and 1,458,443 requests excluding 
Fulltextreq, Nonfree and CGI link requests 
were logged. The distribution of use over the 
user groups is fascinating and, possibly, not 
what might have been expected. Thus, while 
non-subscribers made up 56 per cent of users 
they actually only accounted for about 22 per 
cent of use (Figure 3). The majority of users 
accounted for the minority of use. Those 
opting for the trial made up 7 per cent of users 
but accounted for a sizeable 18 per cent of 
requests. Deal subscribers made up 35 per 
cent of users and accounted for the majority 
(57 per cent) of use. Plainly, just two groups — 
trial non-subscribers and Deal subscribers, 


Figure 3 Percentage of use (articles, abstract, content requests) 


x by type of user 
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generate a lot of site activity — about 
75 per cent of it. 


Use over time 

Change is a major concern of the long-term 
project but clearly one month’s worth of data 
is insufficient for the purpose (we shall 
evaluate further months as the research 
proceeds). However, there is enough here to 
discern hourly and weekly patterns of use — 
the biorhythms of the academic digital 
consumer. Figure 4 shows the number of all 
daily requests (activity) for June 2002. The 
average number of daily requests (article, 
abstracts and content pages) was 51,010 
requests and there were 13,121 articles 
downloaded on average per day. For 
comparison Figure 5 provides an analysis of 
useful requests only. The patterns are very 
similar, if not identical. Figures 6 and 7 give a 


Figure 4 Daily plot of use activity (all requests) 
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Figure 5 Daily plot of use activity (article, abstract and content requests) 
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break down for the individual kinds of 
requests — something that is taken up more 
fully in the section “Type of request”. 

The distinct, rhythmic patterns over time of 
both daily users and use is similar to that found 
in other digital use studies we have conducted 
(Nicholas et al., 1999; 2002a, b). In this way the 
academic consumer is no different than any 
other kind of digital consumer. The low points 
record the relatively low usage on Saturdays 
and Sundays compared to weekdays. Sunday 
recorded just under half the use compared to 
midweek use. This could be because most 
people have access to the system at their places 
of work or study, or simply because people 
prefer not to spend their weekend working or 
studying. The former is probably more 
plausible, but qualitative work will confirm. 


Figure 6 Daily plot of use activity broken down by article, abstract and 
content requests 
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Figure 7 Daily plot of use activity as percentages broken down by article, 
abstract and content requests 
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Figure 8 sheds more light on the topic, by 
showing the distribution of use over day of 
week. Approximately 9 per cent of all weekly 
use occurred on a Sunday, substantially less 
than the 16 per cent of use recorded for 
weekdays. Use fell off on Friday to 14 per cent 
and Saturday also recorded a lower figure of 12 
per cent. Wednesday and Monday recorded 
the highest daily figure (17 per cent). Previous 
studies (Nicholas et al., 2001b) also found 
Wednesdays to be the most popular day to 
view Web-based information. 

Figure 9 gives the distribution of use over ~ 
hour of day (Greenwich Mean Time). Most 
use is recorded from about 9am to 3pm — 42 
per cent of use occurs between these times. 
There is no obvious reason to believe that Big 
Deals change the hourly or weekly patterns of 
information consumption, but it is worth 


Figure 8 Percentage of use distributed over day of week 
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testing this assumption. Figure 10 compares 
hourly use by subscriber type. Deal 

/ subscribers have a much more pronounced 
midday peak and use increases about three 
fold, and probably reflects the extensive 
number of UK Deal users logging on at about 
this time. Trialists and non-subscribers have 
similar patterns of use and might indicate that 
these users are the same type of user. Non- 
Deal subscribers appear to spread their use 
throughott the day, although the fact that 
there is not a lot of use coming from this 

a group flatcens the distribution somewhat. 


Type of reauest 

Figure 11 shows the distribution of use by type 
of request. Well over one in four requests (29 
per cent) were for an abstract, 23 per cent of 
requests were for a “fulltextreq” (this is a 


Figure 10 Use by hour of day by subscriber type 
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“pre request” for an article), and 15 per cent 
for an article. One in ten requests related to the 
table of contents for an actual journal, which 
delivers the contents for many volumes 
(WWW contents) and 5 per cent of requests 
were for the table of contents for an individual 
issue. So 15 per cent of requests in all are for 
contents pages. Of requests, 9 percent were 
related to a CGllink, which is an article 
reference and other type of internal link 
requests. 

The above analysis is plainly influenced by 
the fact that before you view a full-text article 
you have to see the abstract first. 

Whether Big Deal users, with their greater 
choice and freedom of access, do anything 
different when online is clearly of great 
interest. Figure 12 shows the distribution of 
request by type of user and Figure 13 gives the 
same data but as a percentage bar chart. In 
absolute terms Deal subscribers make the most 
use of the site. In looking at the percentage bar 
chart it can be seen that non-subscribers tend 
to make more use of issue and content pages 
by some margin — they do not have automatic 
or upfront access to full-text articles, of course. 
Articles they do view are restricted to journals 
available free on the site made available via the 
Journal of the Week offer or for which they are 
willing to pay. Non-subscribers, aware no 
doubt of their limited access, are probably 
using content and abstracts to screen articles 
and carefully select their downloads, for which 
they then pay. Abstracts also feature strongly 
in the requests of trialists, non-Deal and Deal 
subscribers, though articles were equally 
important. Deal subscribers had the highest 
percentage share of abstract and articles and 
made up two-thirds of their told requests. Deal 


Figure 12 Distribution of request by type of user 
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Figure 13 Percentage distribution of request by type of user 
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subscriber users were least likely to make use 
of content and issue information. There are a 
number of possible explanations for this. 
Possibly Deal subscribers regularly return to 
the same journals and hence do not have the 
need so much to see tables of contents. 
Alternatively Deal subscribers may well use the 
search facility, regular users are more likely to 
have invested the time in learning how to use 
the search facility as this provides a quicker 
way to finding articles. It could also be that 
student users prefer the search engine 
approach. The second stage of the project will 
see the examination of search logs, which are 
held on a different server. 


Number of visits 

In the wider world of the promiscuous digital 
consumer (a world full of choice), it is said 
that the more times a user returns to the site 
the more likely they are to be satisfied with it. 
Coming back to a site appears to constitute 
conscious and directed use — as good an 
approximation of this as you are likely to get 
from Web logs. All this makes return visits a 
powerful performance indicator. The industry 
calls this site “stickiness”. But how frequently 
they return will depend on the nature of the 
site, and it is not clear what would constitute a 
“good” frequency for a journal site. However, 
almost by definition in the case of academics, 
one would have thought that subscribers 
would naturally develop a repeat behaviour in 
order to fulfil their current awareness needs. 
This raises the interesting issue of whether 
logs could be employed to monitor a key part 
of an academic’s role, certainly in the case of 
“old-university” academics who have a 
contractual commitment to conduct research. 
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Table II provides a breakdown of the 
pattern of returnees over the month. Nearly 
nine in ten users visited the site just once in F 
the survey period. Approximately 10 per cent 
of users visited between two to five times and 
half a per cent visited six or more times. This 
is probably not a surprising outcome given the 
limited date range of the survey. Emerald’s 
own research shows that people do visit more 
frequently than once a month but that June 
data is probably a low point because of the 
end of the academic year. Further research 
will shed more light upon this. 

Of course, at present we cannot really be 
certain that it is the same people who visited, 
that is, if indeed these were repeat visits? 
Again, if it is somebody logging in from the 
library, or perhaps from the computer centre 
there might be a problem. We could be more 
certain with academic staff who have their 
own computers. 

Figure 14 breaks down the number of repeat 
visits by the type of subscriber. Non- 
subscribers were least likely to revisit the site, 
which is not surprising — they have less to gain 
from a repeat visit (they have not paid upfront 
for access) and they are also probably D 
opportunistic and promiscuous users. Over the 
month just 4 per cent of non-subscribers 


Table II Visits within a month 


Percentage 
Visit on one day 89.1 
Visited 2-5 10.3 
Visited 6-15 0.6 
Over 15 times None 
100.0 


Figure 14 Number of visits (article, abstract and content page requests)" 
by type of user 
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revisited the site two or more times. Trialists 
were the most likely type of user to return; 

28 per cent of trialists visited the site two or 
more times. The fact that they only had a 
month’s trial meant that their opportunity to 
revisit was limited, so they appear to have made 
hay while the sun shines, but five articles hardly 
signifies a lot of “sun”. Plainly, they were also 
getting acquainted with the site. The 
percentage of people revisiting the site two or 
more times increased to approximately 20 per 
cent in the case of subscribers (non-Deal and 
Deal subscribers). Having a Big Deal made 
little difference. Maybe the number/range of 
journals available to a user is not an important 
current awareness driver. Choices are obviously 
prescribed in the journal world. Thus, to take 
an extreme example, people interested in 
librarianship are unlikely to be interested in 
pigment resin technology. 


Level of activity on a visit/during a session 

Like return visits, the average number of 
requests made in a session is a powerful metric, 
which shows how active, interested or busy 
(confused, perhaps) the user is. It also shows 
commitment. Figure 15 shows the percentage 
distribution of requests (article, abstract and 
content requests) in a user session by 
subscriber type. Approximately 87 per cent of 
non-subscribers made three or fewer requests 
in a session. The fact that they have more 
constrained facilities — the article download is 
more restricted to them, partly explains this. It 
might also be in part due to the fact that they 
are not so practised in moving around the site, 
as they are less familiar with the site because 
they visit less frequently. This conforms to the 


figure 15 Average number of requests (articles, abstract and content 
` page requests) in a session by subscriber type 
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earlier finding, which showed that, while non- 
subscribers made up about 55 per cent of the 
users of the site, they actually accounted for 
only 22 per cent of requests. Some people land 
at the site, visit a couple of pages to see what is 
there and leave. We have come across such 
people before in our studies of the media and 
health — thousands and thousands of them, and 
in our previous Web studies we have labelled 
them “bouncers” (Nicholas ez al., 2001b). 

This contrasts markedly with the situation 
in regard to trialists, 25 per cent of whom 
made three or fewer requests. They were also 
the group most likely to make 11 or more 
requests in a session — about 42 per cent of 
this group did so. Evidently trialists are busy 
in all kinds of ways, they visit most frequently 
and they make most requests when they are 
there. In total 27 per cent of non-Deal 
subscribers and 31 per cent of Deal 
subscribers made three or fewer requests; and 
32 per cent of Deal subscribers made 11 or 
more requests in an average session. This was 
true of 30 per cent of non-Deal subscribers — a 
small difference. Having more journal articles 
available to download might have been 
expected to increase activity levels more, but 
this does not appear to be the case. 

No trialist or subscriber had sessions where 
only a content page was viewed. That is, both 
types of users all went on to view at least an 
abstract request. In total, over a quarter (27 
per cent) of all sessions were sessions in which 
only an abstract was viewed and no article 
downloaded — most people bought what they 
saw in the digital shop window. All trialists’ 
sessions were sessions that resulted in at least 
one article being viewed, while approximately 
20 per cent of subscriber sessions (Deal and 
non-Deal) were sessions where no article was 
viewed, This might point to a form of browsing 
where abstracts are checked but the associated 
article was not looked at. In total, 80 per cent 
of subscriber (Deal and non-Deal) sessions did 
go on to view at least one article. 

Well over half (58 per cent) of all non- 
subscriber sessions resulted in an issue or 
journal content request — these users also 
accounted for all content-only request sessions 
(i.e. where that is the only request made). Of the 
42 per cent of non-subscribers who did go 
further into the site, 88 per cent viewed at least 
one abstract, but not an article, and 12 per-cent 
viewed an abstract and an article. Either by 
paying to view the article or viewing a free article 
under the Journal of the Week article offers. 
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Time metrics 

When vendors charged by time online, time 
was a key use metric but of course most Web 
sites do not employ this charging method and 
so it has lost some — but not all — of its 
potency. Time, after all, is still a precious 
commodity for the user. Table TI gathers a 
number of specific time-use metrics together: 
all requests per hour; article, abstract content 
requests per hour; and article requests per 
hour. On average 3,077 requests were made 
per hour, of which 1,907 were articles, 
abstracts and content page requests and 475 
were article requests. 

Table IV provides overall estimates of 
average page view (or download) time 
comparing time spent on articles and 
abstracts. A skewed distribution of page view 
time was observed and is indicated in the 
table by the differences between the 
arithmetic mean and the median. In such 
cases the arithmetic mean will be biased and 
should not be relied upon. To compensate for 
this the robust estimators, the 5 per cent 
trimmed mean, and Huber’s M-estimator 
were generated. Both give estimates of the 
mean average that are not sensitive to the 
underlying frequency distribution and give 
unbiased estimates of the mean. The 5 per 
cent trimmed mean does this by discarding 
the lowest and highest 2.5 per cent of the 
values and then computing the mean of the 
remaining values, Huber’s M-estimator is a 
weighted mean estimate where extreme values 
are given less weight. 

Articles recorded the longest view time of 
just less than two minutes; abstract view time 
was about a quarter of this and was just over 
half a minute. The average for all requests was 
just under half a minute — 24 seconds. We are 
probably comparing different things here as, 
presumably, few users are “reading” the 
article (although scanning in HTML is quite 
popular as we shall see later), whereas they 
are probably reading the abstract. 
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Table IV Estimates for average view time for articles, abstracts and 
all views 


Articles Abstracts All 
Mean 4442 173.1 176.5 
Median 97 31.0 20.0 
5 per cent trimmed 170.2 53.23 475 
Huber’s M-estimate 111.2 35.14 24.4 


Table V gives average view time for just articles 
by type of subscriber. There are significant 
differences between subscribers. In particular 
Deal users appear to spend twice as much time 
viewing an article compared to non-Deal users. 
This difference may be due to unrestricted use 
available at universities. Universities are more 
likely to be Deal users and these users may 
prefer to read articles online. However, this 
difference may partly result in the use of PDF 
and HTML formats to view documents, rather 
than differences in interest or satisfaction. 
Future qualitative work might tell us when 
scanning becomes reading. 


Number of journals viewed by subscriber type 
All site users can view the abstracts and 
content pages of all Emerald journals — over àj 
150 of them. Figure 16 shows the number of 
different journals viewed by type of subscriber 
in terms of article, abstract, and content 
views. In fact Deal subscribers were more 
likely to view a greater variety of journals, 
which is of course what the Big Deal is 
supposed to be all about, so they are getting — 
and taking advantage of — what they paid for: 
63 per cent viewed two or more different 
journals; 39 per cent viewed between two to 
five different journals, 20 per cent viewed six 
to 15 journals and 4 per cent viewed over 15. 


~ 


Table V Estimates for average view time for articles by type of subscriber 


Articles Abstracts All 
Mean 381.6 333.5 490.7 
Median 70.0 58.0 117 
5 per cent trimmed 135.4 110.4 197.0 
Huber’s M-estimate 82.1 67.1 132.0 





Table II] Average number of users per hour, requests per hour and article requests per hour 


All requests per hour 


Article, abstract content 


Number of article 


requests per hour requests per hour 





Mean 3,487.6 
Median 2,992.0 
5 per cent trimmed mean 3,386.6 
Huber’s M-estimator 3,076.5 


96 


2,086.5 539.4 
1,792.0 474.0 
2,026.6 527.1 
1,906.7 474.5 
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Figure 16 Number of different journals viewed (article, abstract and 
content views) by subscriber type 
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Figure 17 Number of different journals viewed (articles onl) by 
subscriber type 
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Non-Deal subscribers were more likely to 
view just one journal, 50 per cent did so 


compared to 37 per cent of Deal subscribers. 
The availability of the Deal to users seems to 
have encouraged users to extend the range of 
titles viewed. Trial users again made good use 


LL 


a 


between two to five journals, 23 per cent 
viewed six to 15 different journals and 


7 per cent viewed over 15 different journals. 


Approximately nine out of ten non- 
subscribers viewed just one journal. 


of the site and were the group most likely to 
view two or more journals; 36 per cent viewed 


All users can download articles, but only Big 


Deal subscribers have automatic rights to 


downloac articles from more than a hundred 


of them. Non-subscribers, like all users, can 


view articles freely via the Journal of the Week 


offer (see below), trialists can view just five 


articles while non-Deal subscribers are limited 


by the terms of their particular deal with 
Emerald, usually access to a handful of 


journals. The difference this makes in terms of 


the number of articles viewed from different 


journals is shown in Figure 17. This shows the 
number of different journals viewed by type of 
subscriber for article requests only. Here Deal 


subscribers are far more likely to view articles 


from two or more different journals: 53 per 
cent did so compared to 31 per cent of non- 


Deal subscribers, 40 per cent of trialist and 18 


per cent of non-subscribers. This result is 


Table VI gives the average number of 
different journals viewed per subscriber 


organisation (all users from that organisation 


being bundled together for this analysis) by 


plainly a function of range of full-text access. 
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No. ournals viewed 
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(]5 to 15 

2 to 5 

More 








Non deal subscriber 
Trialist Deal subscriber 


Non subscriber 





subscriber type. On average, non-Deal 
organisations viewed about five different 
journals and on average requested about 12 
articles. While those organisations subscribing 
to the Big Deal viewed approximately 46 
different journals and requested about 93 
articles. Quite a difference: further evidence 
perhaps of the workings of the Big Deal. 
There is a large difference between the use 
figures of the subscriber types. Non-Deal 
users requested significantly fewer articles and 
made use of fewer journals compared to those 
organisations subscribing to the Deal. 


Individual journal titles viewed 

The titles of the journal groups consulted are 
of interest, and the comparisons between them 
of special interest. Table VII gives the top 
ranked journals (according to all requests 
made) by type of subscriber. The top 15 
journals accounted for 60 per cent of journal 
views in the case of non-subscribers (including 
trialists), 38 per cent for non-Deal subscribers 
and 36 per cent for Deal subscribers. The 
lower percentage for Deal subscribers indicates 
that use is distributed more evenly over a wider 
number of journals. It could be that they have 
a greater freedom of choice, or that they are 


Table VI Average number of journals and articles viewed 3er organisation 
for non-Deal and Deal subscribers 


Non-Deal subscriber Deal subscriber 








Articles Journals Artides Journals 
Mean 59.2 11.13 3448 51.6 
Median 10.0 4.0 77.0 44.0 
5 per cent trimmed mean 24.7 8.1 2131 50.1 
Huber’s M-estimate 12.1 54 92.7 46 
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Table VII Journal use by subscriber type 


Non-subscriber use of journals 


% 


Non-Deal subscriber use of 
journals 
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Deal subscriber use of journals 





Integrated Manufacturing Systems ° 
Industrial & Commercial Training ° 

int. Jnl of Operations & Production Man't 
The Jnl of Management Development 
int. Jn! of Public Sector Management ° 
Employee relations ° 

Int Marketing Review 

Industrial Management & Data Systems 
Strategy & Leadership * 

Team Performance Management ° 

Jni of Intellectual Capital 

Reference Services Review 
Environmental Management & Health ° 
Interlending & Document Supply ° 
OCLC Systems & Services ° 


Per cent of use accounted by top 15 
Note: * Journal of the Week title 


10.0 int. Jni of Quality & Reliability Mant 6.2 European Jn! of Marketing 
9.0 European Jn! of Marketing 31 Management Decision 
8.9 Leadership & Organization Development 3.0 Leadership & Organization Development 
5.1. In! of Quality in Maintenance Engin. 3.0 Marketing Intelligence & Planning 
4.7 Jnl of Services Marketing 2.7 Jnl of Knowledge Management 
4.4 Jnl of Knowledge Management 2.5 Jnl of Consumer Marketing 
4.2 Accounting Auditing & Accountability Jnl 2.3. Int. Jnl of Contemporary Hospitality Man’t 
4.2 int Jnl of Operations & Production Mant 2.3 Jnl of European Industrial Training 
3.9 Jnl of Organizational Change Man'‘t 2.0 Personnel Review 
3.8 Int. Jnl of Service Industry Man’t 2.0 int. Jnl of Service Industry Man’t 
2.0 | Management Decision 1.9 Internet Research: Networking Appls 
1.8 Jnl of Consumer Marketing 1.8 Jnl of Product & Br & Management 
1.8 The TQM Magazine 1.8 The TQM Magazine 
1.5 Marketing Intelligence & Planning 1.7 Jnl of Services Marketing 
1.2 Jnl of Managerial Psychology 1.7 Managing Service Quality 
60 Per cent of use accounted by top 15 38 Per cent of use accounted by top 15 

















just a different kind of user (lecturers rather 

than students for instance). 

Comparing the titles in the lists of the two 
subscriber groups we can see that two at the 
top of the tree are the same, but otherwise 
there is much difference. The non- 
subscriber list is very different from the lists 
of the two groups of subscriber. In fact, the 
data in column one are skewed by a 
marketing activity called “Journal of the 
Week” which gives free access to two 
specific journals and should be read in this 
context. The journals offered free in June 
were as follows: 

* Week 27-05-2002 — Team Performance 
Management and Strategy & Leadership. 

* 05-06-2002 — Employee Relations and 
International Fournal of Public Sector 
Management. 

* 10-06-2002 — Industrial and Commercial 
Training and Environmental Management 
and Health. 

* 17-06-2002 — Interlending & Document 
Supply and Integrated Manufacturing 
Systems. 

* 24-06-2002 — OCLC Systems & Services 
and Pigment Resin Technology. 


This marketing ploy is so effective that all 
but one of these titles (Pigment Resin 
Technology, which obviously has a very 
specialist readership) appears in the non- 
subscribers’ list of the 15 most used titles 
(see Table VID. 


The use of graphs - article, abstract and 
content page views — for each of the titles are 
shown in Figures 18-22. 

Typically there is a big impact on the use of 
the journal as a result of the journal being 
available free because of the Journal of the 
Week promotion. On average, it increases use 
of the journal by over ten times. Digital 
journal users, like consumers everywhere, can 
spot a good deal when it comes their way, and 
are willing to change (in droves) their 
behaviour as a result. 

Figure 23 shows the percentage use of all 
offer journals within their respective offer 
weeks, use of the same journals in all non-offer 
weeks and use of all other journals. Use of offer 
journals within the offer week make up about 
12 per cent of use, the same journals in all 
non-offer weeks made up 6 per cent of use. 
Figure 24 shows the distribution between offer 
and non-offer journals by type of subscriber. 
As we might have expected offer journals were 
far more likely to be viewed by non-subscribers 
and trialists — 26 per cent of non-subscriber 
and trialist use was made up of views to a 
Journal of the Week title compared to about 2 
per cent of Deal subscribers and 2 per cent of 
non-Deal subscribers. 

In terms of article view time, Journal of the 
Week titles performed poorly compared to 
when they were not being offered and to non- 
offer journals. Table VIII gives article view 
time, broken down by views of Journals of the 
Week in the offer week, views to the same 
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Figure 18 Team Performance Management and Strategy Leadership 
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Figure 21 Interlending and Document Supply and integrated 
Manufacturing Systems 








Figure 19 Employee Relations and International Journal of Public Sector 
Management 
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Figure 22 OCLC Systems and Services and Pigment Resin Technology 








14,000 
12,000 
a: 10,000 
8,000 


6,000 





Employee Relations 


Int. Journal of Public 





Sector Management 
01.06.2002 08.06.2002 15.C6.2002 22.06.2002 29.06.2002 


14,000 


12,000 


10,000: 





OCLC Systems & 


Services 


Pigment & Resin 





OE — d Technology 
08.06.2002 15.06.2002 22.06.2002 29.06.2002 











Figure 20 Industrial and Commercial Training and Environmental 
Management and Health 
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Figure 23 Impact of offer: ten journal offer weeks compared to non-offer 


weeks and non-offer journals use 
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Figure 24 Distribution of Journal of the Week (article, abstract and 
content links) by type of subscriber 
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journals outside of the offer week and for non- 
offer journals. On average Journal of the 
Week articles within offer weeks were viewed 
for about 46 seconds (Huber’s estimate) — less 
than half the view time recorded for the same 
journals viewed not in an offer week, and just 
about a third of the view time of all non-offer 
journal articles. This argues that, although the 
offer of the week does attract lots more use, 
much of this use is of a squirreling sort. 

We can look at the use of journals from 
another perspective — by calculating how much 
time is spent “reading” them. Table IX shows 
the ranking of journals — the top and bottom 
ten — by article view time. There is 
approximately two and half minutes difference 
in page view time between the top (Australian 
Journal of Hospitality Management) and bottom 
journal (Microelectronics International). 

Five of the bottom ten journals ranked by 
view time were Journal of the Week journals, 
which we know attract bouncer users who 
squirrel the data away. Furthermore, four of 
the top ten ranked by view time are only 
available in a PDF format. The longer page 
view time here is thought to result from users 
trying to read the PDF format online. This 
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format is harder to read online and hence 

would attract a longer page view time. 

Differences in page view time seem to be 5 
related to the online format and if the journal 

comes under any offer — like the Journal of the 

Week offer. Other factors that might account 

for page view time difference are interest and 

size and make up of the journal. 


Volumes/years of a journal consulted 

The average number of back volumes viewed 

per journal was just under eight with more 

recent volumes being viewed the most. The 

range of volumes from which articles were -= 
downloaded was less than this and the average 

was about seven issues (6.7). 

Of course, volume use tells us something 
about the age of material consulted, 
obsolescence etc. Given the fact that the 
archives on the Emerald archive seldom go 
back further than ten years, the data obtained 
will only be partial. However, it might be 
expected that with improved access to journal 
archives journal back numbers might 
increasingly be used, through improved 
exposure, etc. 

Figures 25-27 show the distribution of 5 
articles used by year for three of the top 
performing journals by article use. The three 
journals are Industrial and Commercial 
Training (online from 1994-), European 
Journal of Marketing (1993-) and Integrated 
Manufacturing Systems (1994-). In terms of 
use, most use was made of the most recent 
years, with use for earlier volumes decaying 
over time. However, there was a marked 
difference between the three journals. 

The most recent two years made up about 
27 per cent of use for the Eurepean Journal 
of Marketing. However, this percentage 
increased to 42 per cent and 47 per cent, 
respectively, for Industrial and Commercial 
Training and Integrated Management Systems, 
with the latter decaying fastest of all in 

use terms. 

Figures 28 and 29 give the distribution of 
abstract, article and content back issue views 


Table VIII Article view time by views of Journal of the Week in offer week, out of offer week and for 


non-offer journals 


Non-offer journals 


Offer journals in offer Offer journals in non- 





Mean 486.6 
Median 111.0 
5 per cent trimmed 191.2 
Huber’s M-estimator 126.0 


weeks offer weeks 

215.6 401.6 
40.0 91.0 
NG 157.9 
46.0 103.7 
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Table 1X Top and bottom ten journals viewed ranked by view time 





Top ten journals by page view time Seconds 
7 í Australian Jn! of Hospitality Management ® 288 
Records Management Jnl ° 169 
Jnl of Documentation ° 167 
Jnl of Fashion Marketing Management ° 154 
The Jnl of Business & Industrial Marketing 153 
Online Information Review 151 
Policing: An Int. Jnl of Police Strategies & Mant 151 
Int, Jnl of Service Industry Management 151 
Jn} of Educational Administration 149 
Reference Services Review 149 


for the European Journal of Marketing broken 

down by type of subscriber. Again use of back 
issues decays with age of issue. The breakdown 
in use of back issues between subscriber type is 


Figure 25 Number of articles viewed by year — Industrial and Commercial 
Training (1994-) ; 
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ig Figure 26 Number of articles viewed by year — European Journal of 
Marketing (1993-) 
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Bottom ten journals by page view time Seconds 
COMPEL: Int. J. Comput. & Maths in Electr. 67 
Team Performance Management ° 66 
Int Jn! of Public Sector Management ? 58 
Integrated Manufacturing Systems ° 53 
Industrial & Commercial Training ° 53 
Industrial Lubrication & Tribology 48 
Environmental Management & Health ? 47 
Pigment & Resin Technology 45 
Interlending & Document Supply 44 
Microelelectronics int. 27 


Notes: ĉ Journal of the Week; ° Only available in a PDF format 


proportionally the same and there is little 
evidence that Deal users use back issues more 
than non-Deal users and non-subscribers. 
However this needs to be confirmed by 
extending the analysis to other journals. 


Figure 27 Number of articles viewed by year — Integrated Manufacturing 
Systems (1994-) 
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Figure 28 Abstract, article and content back issue views for the 
European Journal of Marketing broken down by type of subscriber 
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Figure 29 Abstract, article and content back issue views for the European example, that the ratio of non-subscriber 
Journal of Marketing broken down by type of subscriber as a percentage use to Deal subscriber use impacted on 
bar chart overall article use. Figure 31 explores this 


relationship. There is evidence of auto 

correlation{[5]. However, it does seem 
that journals with a higher percentage of 

i 
40 cl) H mee 

iF, Ja 
$ aL =æ 
average number of requests and article 
requests, and the average article view time. 
85 
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Subject analyses 

Emerald classifies its business and 
information studies journals into 22 subject 
sub-categories — some journals belong to a 
number of categories. Figures 32 and 33, 
respectively, show for each category the 
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Deal subscribers’ use also had a higher 
article use. 

Non subscribers On average training journals recorded the 
pa aa highest number of requests and article 
requests but one of the lowest average page 
view times of just over a minute. Professional 
development journals recorded both a 
relatively high request score and a relatively 
high article view time score. Human resource 
management and hospitality management 
score well on the total number of requests. 
Library collection development scored 





81 





Articles read by extent of journal archive 

A relationship was found between the 
numbers of years for which the journal was 
available online and the number of articles 
requested. This is shown in Figure 30 and 
implies in part that journals with longer 


electronic archives perform better than highest on view time, more than twice the 

new journals. time for environmental management. There 
A number of factors are likely to impact are real differences here that might be 

on the number of articles requested. For explained by length of article, size of journal 

example, the number of issues per year is and type of content. 

likely to have an impact as well as age of the Figure 34 shows the number of different 

journal, and there may well be additional subjects viewed by type of subscriber. The 


factors. A study of the variables found, for data prove most interesting. Trial 


Figure 30 Relationship between the number of articles requested and the Figure 31 Ratio of non-subscriber use to Deal subscriber use and the 
number of years for which the journal is available online number (log) of articles read 
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Figure 32 Average number of requests by subject category: all requests and article requests 
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Figure 33 Average article view time by subject category — articles only 
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Figure 34 Number of different subjects viewed (article, abstract and 
content page requests) by subscriber type 


subscribers typically only viewed one subject 
area: 89 per cent did so, compared to 56 per 
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cent of Deal subscribers and 42 per cent of 
non-Deal subscribers who did so. There is a 
difference between Deal and non-Deal 
subscribers. Deal subscribers were more 
likely to view a greater number of different 
subject areas. Nearly one in three Deal users 
looked at journals from three or more subject 
categories, however this was only true of 26 
per cent of non-Deal subscribers. 
Surprisingly, there was not a big difference 
between trialist and Deal subscriber subject 
use, and there appears no easy explanation 
for this. Further research needs to clarify if 
these users are searching journals using the 
search facility, as these users seem likely to 





look at more journals. 
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Viewing and downloading formats 

The user’s preference for viewing online or 
reading offline sparks many a discussion. On 
Emerald users can choose to view files either 
in a PDF or an HTML format. There are a 
number of reasons why users might prefer one 
format to another. Research students may 
prefer to print out a copy in the style of the 
original and may prefer that format to refer 
back to while others might wish to read it 
online or prefer the flexibility of viewing the 
article in an HTML format. 

Users could in fact choose between four 
types of PDF format: to view the whole 
document in PDF (Doc PDP); to view a page 
in PDF (Page PDF); <o view a directory in 
PDF (Dir PDF); and to view a file in PDF 
(File PDF). Approximately 10 per cent of 
journals were only available in a PDF format. 
This occurs as the decision to make a journal 
available in an HTML format is made after 
the decision to make available a journal in a 
PDF format. In total HTML made up 38 per 
cent of all views while PDF made up 62 per 
cent, however, the difference between 
documents viewed as PDF and HTML is 
much smaller: 40 per cent compared to 38 per 
cent (Figure 35). 

Figure 36 provides tne breakdown of choice 
of article view format by type of user (the figure 
excludes views to artic.es that could “only” be 
viewed in a PDF format). When given a choice 
as to what format to view an article less than 
half of Deal users (45 per cent) selected to view 
in an HTML format; this was only true of a 
quarter of non-Deal subscribers, a third of 
trialists and about a fifth of non-subscribers. It 


Figure 35 Distribution of format view PDF vs HTML 
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Figure 36 Format article view by type of subscriber 
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appears that, on average, non-subscribers and 
non-Deal subscribers prefer a PDF format, 
while Deal subscribers are more willing to select 
an HTML format. Perhaps Deal subscribers 
are more likely to be experienced IT users, are 
used to reading online and managing 
information in a digital format. It could be that 
people access HTML if they expect to scan but 
PDF if they expect to download, print out and 
keep. There is some evidence that even the 
most sophisticated academic community 
(astrophysicists) access PDF and print out to 
read or perhaps file (Boyce and Tenopir, 2002). 
Future research needs to verify this. 

A difference in the download time between 
the formats was found. It took approximately 
seven times longer to download the same 
article in a PDF form compared to an HTML 
form: 14 seconds compared to two seconds. 
The PDF format delivers the article in the 
style of the published article and requires the 
client machine to have a PDF reader. The 
additional download time results from the 
client machine loading up the PDF reader 
and displaying the article. HTML requires no 
additional software other than the browser 
software that the user is currently using and 
there is no delay in loading an additional 
program. 

Table X gives the view time comparing all 
articles, PDF views and HTML views. 
Although PDF files took longer to download, 
HTML files recorded a longer page view time 
by over half as much again compared to PDF 
views: 142 seconds to 94 seconds (Huber’s 
M-estimate). PDF is actually more difficult to 
read on screen and although some users may 
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Table X View time comparing all-articles with articles viewed in a PDF 


and HTML format 


í 


4 


PDF HTML 


All articles format format 





Mean 
Median 


444.2 390.0 534.9 
97 81 126.0 


5 per cent trimmed 170.2 146.6 212.0 
Huber's M-estimate 111.2 93.7 142.0 





well be reading PDF on screen it seems more 
likely tha: PDF requested articles are being 
printed. HTML formatted articles are easier 
to read and manipulate on screen and 
although some users will print off on an 
HTML format it seems likely that users — 
those recording an above average HTML 
view time — are actually reading the article 
online. Experienced users will be selecting 
parts of the article to read while “skim” 
reading other parts. 

Choice of format may well vary with 
institution. Users in some institutions — like a 
library — may well not be able to work “on- 
line” and hence opt to print out a PDF 
formatted file. Cost might also come into it — a 
PDF formatted file is likely to be cheaper to 
print than an HTML file as the PDF formatted 
file is usually displayed on fewer pages. Again 
further research is needed to clerify the impact, 
if any, of how people use each format and if 
institution and cost is an issue. 

Earlier it was pointed out that Deal 
subscribers were more likely to request 
articles in an HTML format and we would 
expect those users to have a longer article 
page view time. It was found that Deal 
subscribers had approximately double the 
view time compared to non-Deal subscribers: 
131 seconds compared to 67.1 seconds. 

We might expect the variation in choice of 
format to vary with age of user, experience, 
type of research and characteristic of user 
organisation. We shall investigate age at a 
later date, but Figure 37 gives the breakdown 
of format choice of research or laboratory type 
users, by whether the research centre was 
commercial, educational, or government. All 
business research laboratories were non-Deal 
subscribers, while all academic and 95 per 
cent of government laboratorizs tended to be 
Deal subscribers. 

For this small group of users, who 
accounted for about 0.1 per cent of use, 
business research users were more likely to 
view an article in an HTML format, 65 per 
cent of these users did so compared to 39 per 
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Figure 37 View format by organisation type — research organisations only 
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cent of educational users and 22 per cent of 
government research users. Future qualitative 
research is needed to examine reasons for 
these differences. 


Categorising users (and use) in other ways 
Country and organisational affiliation of user. 
So far we have been looking at use in regard 
to subscriber type only, but it is possible to 
analyse use by country of origin, by type of 
organisation to which they are affiliated and 
by individual named institution. Figures 38 
and 39, respectively, break down use by the 
country visitors came from and their 
organisation type, as indicated by their 

IP address. Almost half of use came via 
USA IP addresses and 20 per cent via UK 
ones. IP location addresses are renowned 
(Nicholas eż al., 1999) for being incorrect, 


Figure 38 Percentage breakdown of use by country of IP address of user 
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Figure 39 Percentage breakdown of use by organisation of IP address of user Figure 40 Percentage breakdown of user by country of IP address of user 
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as many UK users will have a USA domain 
name address. Thus we should compare 
these percentages to the known geographical 
distribution of Deal subscribers derived 
from address details rather than the IP 
number. Viewed this way less than 10 per 
cent of use by Deal subscribers came from 
the USA while 40 per cent from the UK, 18 
per cent from Australasia and 17 per cent 
from the Far East. The understatement by 
IP location address occurs because many 
non-US organisations, mainly commercial 
and Net providers, prefer a USA registered 
IP address and these users will in fact come 
from other countries. However, the IP 
country location analysis is useful as location 
can be resolved for all users — non- 
subscribers, non-Deal subscribers as well as 
Deal subscribers and secondly the type of 
user organisation, which is also IP address 
specific, can also be analysed. 

In total, 46 per cent of use was accounted 
for by people who accessed the site via an 
academic institution, 40 per cent via a 
commercial company and 12 per cent via a 
Net provider (home users, perhaps). Note — 
Net provider organisations generally prefer to 
register as a “.com” organisation and this use 
and Net use may well reflect use by 
individuals or small organisations. Just about 
seven in ten Deal subscriber use and eight in 
ten non-Deal subscriber use was identified as 
coming from academic institutions. 

By way of contrast Figures 40 and 41 give 
the same information for users and as can be 
seen there is not a lot of difference between 
the breakdown between use and users by 
location and organisation type. 











Percent 








Figure 41 Percentage breakdown of user by organisation of IP address of use 
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In a cross-classification it was found that 85 
per cent of commercial and 70 per cent of Net 
organisation use was also attributed to USA 
use. However, this is unlikely to be the case. 
Take, for example, the case of 

BT connect.com, users accessing the Web via 
this provider would be classified as a USA 
commercial user while in fact they would be 
likely to be UK users accessing the Web via a 
Net provider. 

We are more sure that accesses by academic 
organisations are by academic organisations. 
Just over a third of use (38 per cent) by 
academic organisation came via the UK, 27 
per cent came via the USA, and 26 per cent 
came from academic institutions based in 
Australia and New Zealand. 

Figure 42 shows use by the country visitors 
came from, as indicated by their IP address, 
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Figure 42 Site use by location of user by subscriber type 
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Figure 43 Site use of organisation affiliation of user by su2scriber type 
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broken down by subscriber type. A higher 
proportion of non-subscriber use was 
identified as coming from the USA and 
Eastern Europe compared to Deal and non- 
Deal subscriber use and less non-subscriber | 
(including trialists) use was identified as 
coming from the UK. Only 10 per cent of 
non-subscriber (including trialists) use 
originated from the UK, 43 per cent of use by 
this group came from the USA and 20 per 
cent from Eastern Europe. We might have 
expected the Eastern Europe figure on the 
grounds of cost and lack of resource, but the 
US figure is more puzzling. In comparison 


` only 28 per cent of Deal subscriber use came 


from the USA and about 14 per cent from 
Eastern Europe. Most subscriber use, nearly a 
third, came from the UK. The high 
percentage coming from the USA probably 
reflects individual users accessing the Web via 
UK based Net provider organisations but 
having USA IP registered addresses. As can 
be seen (Figure 43) nearly eight in ten non- 
subscriber use was via commercial and Net 
provider IP identified organisations (note 
many Net providers will have a commercial IP 
address). We would expect home users to use 
an IP provider and these users are likely to be 
non-subscribers. 

Looking at country more closely (Figure 
43), Deal subscribers seem particularly strong 
in Australia and New Zealand, 18 per cent of 
users are identified as coming from Australia 
compared to 7 per cent of non-Deal 
subscribers and 7 per cent of non-subscribers 
(including trialists). A high proportion of 
non-Deal subscribers come from Malaysia 
and Singapore, while Canadian users seem 


surprisingly low though Canadians made up 
10 per cent of all non-subscribers. 


Conclusions 


The Emerald data has proved very 
interesting. The Emerald service is open to 
virtually anyone with an interest. We can 
compare and contrast the behaviour of four 
distinct user groups — all with different levels 
of access and we are beginning to shed light 
on the question “Do people with access to 
more journals have fundamentally different 
information seeking patterns as evidenced by 
the logs”. Of course, as yet, we do not know 
whether these people are part of the Big Deals 
of other publishers. 

It is too early to make any hard conclusions, 
but it is plain that much of this deep log data is 
unique and that publishers and participating 
libraries (never mind users) have not seen the 
like of what we are attempting to do. 

To recap on our early findings: 

+ There is a lot of site activity. There were 
approximately 208 people using the 
system per hour. They made an average 
of 3,077 requests of which 475 were 
article requests. 

* Non-subscribers accounted for 45 per 
cent of abstract requests, but just 37 per 
cent of article requests. 

+ Articles recorded the longest view time of 
just less than two minutes; abstract view 
time was about a quarter of this and was 
just over half a minute. View time here 
may well be the time the user spends 
scanning the document. 
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¢ Subscribers were the more frequent 
visitors. A total of 19 per cent of 
subscribers visited the site two or more 
times in June but this was only true of 4 
per cent of non-subscribers. 

* Deal subscribers differed from non-Deal 
subscribers in that they had a longer 
article view time, viewed more journals 
and viewed a greater number of journals 
from different subjects. 

* Trialists proved to be active users of the 
site. 

* Nearly eight in ten of non-subscriber use 
was via commercial and Net provider IP 
identified organisations (many of these 
now have commercial registered IP 
addresses). We would expect home users 
to come in via a Net provider and to be 
non-subscribers. 

* Some of the biggest visitors were not 
users as such but robots that index the 
Net, Net providers and machines that 
cache site content. It would be sensible 
for Big Deal vendors to adopt a robot 
excluding convention. 

» Big Deal users take advantage of the 
greater choice on offer. On average 
during the one-month survey period 
non-Deal subscribers viewed about five 
different journals and on average 
requested about 12 articles. While those 
institutions subscribing to the Big Deal 
viewed approximately 46 different 
journals and requested about 93 articles. 
Another way of looking at it is that 
Big Deal users were more likely to view 
two or more journals, 57 per cent did 
so compared to about 30 per cent of 
non-subscribers and non-Deal 
subscribers. 

« Users chose to view articles in an HTML 
format 38 per cent of the time. Deal 
subscribers were more likely to view 
articles in an HTML format. The shorter 
view time of PDF suggests that users are 
printing off with PDF and scan reading 
with HTML. 


Ironically, while we sought to shed light on 
the Big Deal users and what resulted from the 
greater choices open to them, probably the 
most interesting data to emerge concerns 
trialists. This shows clearly that user 
behaviour can be easily manipulated through 
the use of commercial incentives — incentives 
all consumers, online and offline, are 
familiar with. 
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Finally, a word of caution — “hits” data 
should not be used to define academic 
behaviour until we know more. Thus, in the 
print environment we know that the main 
users of library facilities are students and it is 
likely that the main users of e-journals are 
students too. The implications are significant. 
For example, whereas it is likely that an 
academic will have a view on the relative 
reliability or authority of journals, students 
(even graduate students) are thought not to 
have such a view and they may well just be 
using key words to determine which journals 
they use. Also it could be that Emerald data is 
biased towards management and library 
scientists who are very much practitioners on 
the whole or close to practitioners. They may 
well have a different approach to (say) 
historians or theoretical social scientists like 
anthropologists. 


Notes 


1 The Web, the kiosk, digital TV and the changing and 
evolving face of consumer health information 
provision: a national impact study, Department of 
Health, 2000-2002; Web log analysis: case study 
newspapers, News International, 1998-1999. 

2 Project funded by the Ingenta Institute, 
July-December 2002. 

3 Although change is not guaranteed as Boyce and 
Tenopir have (2002) found in regard to the claims of 
physicists, that by now no one would be printing 
out to read but would read on screen. This has not 
come to pass yet. 

4 Historically, in fact, Internet hits were the count of 
all downloads — i.e. the total number of the multiple 
elements of a “page”; then the practice changed 
with hits resolved to pages by excluding multiple 
elements. 

5 Information of the dependent variable is also part of 
the independent term — in this case the independent 
term is the dependent term broken up into two 
variables and expressed as a ratio. 
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Note from the publisher 


Emerald Fulltext 2003 launched 


We are pleased to announce the launch of the 
latest version of Emerald Fulltext, providing 
even more sophisticated, online access to 
leading management and LIS journals. 

Emerald Fulltext 2003 delivers more 
journals than ever before. The addition of 16 
new titles brings the total number of titles 
delivered by Emerald Fulltext to a market 
leading 158. The total archive of articles is 
now over 40,000. 

The 2003 release incorporates a host of new 
features designed to make user searches even 
more effective. Users will now be able to 
specify the journals they wish to be included 
in a search and then refine that search by 
adding or removing journals. Searching is 
now also supported by comprehensive 
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context-sensitive help. There is also a 
re-designed interface, with hyperlinks to all 
new help pages. Once a search is complete, a . 
new “search result customisation feature” 
empowers users to select those articles from 
within the results that they wish to receive by 
e-mail. 

Another significant usability feature, aimed 
at encouraging users to exploit Emerald 
Fulltext 2003 to the full, is a highly intuitive 
browse menu. Through the extensive use of 
clearly defined options, the in-depth browsing 
of journals is now both simple and 
straightforward. 

Amongst the key features are user definable 
subject search alerts. These empower users to 
define specific areas of interest, to which they 
will be alerted whenever work in that area is 
published. Easily updateable, this new feature 
will keep even the most demanding researcher 
abreast of all newly published work in their 
particular area of specialisation. 

Emerald Fulltext 2003 can also identify 
special issues of journals, where the journal 
focuses on a particular subject, making them 
easier for the user to identify and select. 

Administrators of Emerald Fulltext 2003 
are now able to set the parameters of all user - 
searches to include only a pre-specified set of 
journals, for example, according to an 
organisation’s particular focus or access 
policy. 

Emerald Fulltext 2003 is available for trial 
by clicking the “try” button on the 
www.emeraldinsight.com home page. 
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_ The online full text management research database 


© Full text access to the latest research and global thinking 


© Prestigious, peer reviewed management articles with user access for all — whenever, wherever 
$ . 


; ©) Sophisticated yet simple searching, with archives that significantly expand your 
library collection‘and training resources to encourage and support usage 


` 


O Further Emerald products are available. Evaluate Emerald 


content for yourself ~ visit www.emeraldinsight.com for your 
-_ 30-DAY FREE TRIAL! 
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Research Register 


Bringing research to the wider community 


©) The Emerald Research Register provides an online forum for the circulation of pre-publication 
information on research studies in management and related fields worldwide 


O Open to all, the Emerald Research Register provides a meeting place for researchers to find 
collaborators for their research projects 


© The Emerald Research Register facilitates the sharing of information with other 
researchers in similar and different fieids 


© Selected abstracts will be published in one or more of our journals, as we are 
committed to linking research work in progress to editors and publishers 


©) We aim to help researchers gain advanced recognition among their 
peers by publicising their work at the earliest possible opportunity 


For further information please contact the Research Register Editor 
at rreditor@emeraldinsight.com 
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Asa guide, articles should be between 2,000 and 8,000 words 
in length. A title of not more than eight words should be 
provided. A brief autobiographical note should be supplied 
including full name, affiliation, e-mail address and full 
international contact details. Authors must supply an abstract 
of 100-150 werds. Up to six keywords should be included which 
encapsulate the principal subjects covered by the article. 

Where there is a methodology, it should be clearly 
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For photographic images (plates) good quality original 
photographs should be submitted. If supplied electronically 
they should be saved as .tif or jpeg files at a resolution of at 
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should be set at the highest resolution/quality as possible. 

in the text of the paper the preferred position of all figures 
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succinct and clear captions for all figures and plates. 

Tables must be numbered consecutively with roman 
numerals and a brief title. In the text, the position of the table 
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in Table IV”. 

References to other publications must be in Harvard style 
and carefully checked for completeness, accuracy and 
consistency. This is very important in an electronic environment 
because it enables your readers to exploit the Reference 
Linking facility on the database and link back to the works you 
have cited through CrossRef. You 
should include all author names and 
initials and give any journal title in 


You should cite publications in 
the text: (Adams, 1997) using the 
first named author's name. At the 
end of the paper a reference list in 
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` For books: surname, initials, 
(year), title of book, publisher, place 
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The manuscript will be considered to be the definitive 
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Knowledge sharing: a game people play 
Alton Chua 


Keywords Knowledge management, 
Game theory, Payback 


Examines the dynamics of knowiedge sharing using 
the multi-person game-theoretic framework. 
Proposes that an individual’s knowledge sharing 
tendency is driven by a set of contextualised 
concerns and interests not unlike the notion of 
payoff in game theory. Furthermore, the decision 
to share or withhold knowledge depends on that 
which yields a higher payoff. With this premise, 
submits two objectives. One is to investigate if an 
individual’s perceived payoff of sharing knowledge 
is contingent on the knowledge sharing behaviour 
of others. The other is to analyse the perceived 
payoff of knowledge sharing and determine if it can 
be characterised by an archetypical game in the 
game-theoretic model. An empirical study was 
conducted among nearly 100 students in a local 
institute of higher education. The scope was 
confined to the students’ willingness to post 
asynchronous entries to an electronic discussion 
forum. Finds that the individual student’s 
perceived payoff of sharing knowledge was 
contingent on the knowledge sharing behaviour of 
others. Furthermore, the perceived payoff of 
knowledge sharing among them could be 
characterised by a multi-person assurance game. In 
conclusion, discusses three implications for 
managers who aim to sustain asynchronous 
knowledge sharing in their organisations. 


Accountability and demonstration of the value of 
information services in South African law firms 


Olwyn Garratt and Adeline du Tott 


Keywords Information, Value, Law, 
Financial accounting, Audit, South Africa 


The value of library and information services in the 
incorporated South African law firm was 
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investigated with regard to accountability to 
corporate management. A cost accounting exercise 
was developed and tested in seven South African 
law firms of varying size. Cost per practitioner was 
calculated in order to correlate data from the 
participating organisations so that average relative 
spending might be determined. Attempts to relate 
the unit cost of library and information services to 
use and usefulness were inconclusive and these 
efforts were unable to demonstrate the value of the 
corporate investment. Recognised business 
practices relating to corporate accountability were 
considered with regard to the relevance to 
corporate support services. These included ISO 
9000 and the practice of internal auditing. 
information services audit was proposed j 
accordance with a recognised manage! 
process. 










First steps towards providing the UK 
care information and advice via their eyision nue . 
sets: an evaluation of four Department oRHeaith 7 Won 
sponsored pilot services 7 


David Nicholas, Paul Huntington, Peter Williams and 
Barrie Gunter 


Keywords Digital, Interaction, Television, 
Health care, Evaluation, United Kingdom 


Provides a summary of a Department of Health 
funded research study investigating the 
performance and impact of four pilot digital 
interactive television services in the consumer 
health field. These were launched in various 
locations in the UK in 2001. Text and video, 
interactive and transactional services were 
featured. Pilots were investigated using a 
combination of research methods. The four pilots’ 
performance varied but overall there was 
sufficient evidence to suggest that consumer 
health digital interactive television has a healthy 
future. 


Library portal solutions 
Andrew Cox and Robin Yeates 


Keywords Libraries, Management systems, 
Internet, Effectiveness 


This article reviews a significant range of new 
products developed by library management system 
(LMS) suppliers, which will be referred to for 
convenience as library portal solutions (LPS) and 
whose purpose is to manage access to e-content in 
an integrated way with books, journals and other 
library stock. The article reviews current 
understanding of user requirements and then 
describes the state of the current market place, 
based on a survey of suppliers conducted in May 
and June 2002, as well as on supplier Web sites and 
live demonstrations. The article concludes by 
summarising evaluation criteria for choosing a 
system, and makes some tentative predictions 
about the development of the market place in the 
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next five years. The perspective is primarily that of 
a UK academic library, but the products are 
certainly of interest to libraries in many sectors in 
every country. 


E-mail overload in the UK workplace 
Jenni Ingham 


Keywords Electronic mail, United Kingdom, 
Problem identification 


Establishes whether e-mail overload exists in the 
UK workplace through a series of in-depth 
interviews. Evaluates the use of e-mail and 
attitudes towards its use in a business environment. 
Concludes that e-mail has recently penetrated into 
organisations as a business communication tool 
and that the level of seniority in a company impacts 
on the number of e-mails sent and received. 
Establishes that e-mail overload exists, although it 
is difficult to say to what extent, as e-mail usage is 
sometimes very personal. Devises a set of 
guidelines to give advice on how to manage 
incoming messages. 
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After the disaster: drawing up the insurance claim 
Robert Howes 


Keywords University libraries, Disaster planning, 
Floods, Insurance, United Kingdom 


Reports on the results of a major flood at the 
University of Sussex Library’s repository at Lewes 
in October 2000, which severely damaged a 
collection of 42,000 books, 14,000 bound 
periodical volumes, microfilms, newspapers, report 
literature and other materials. It describes the way 
in which the detailed insurance claim was drawn 
up and the methodology used to quantify the 
losses, using a combination of machine-readable 
data, card catalogues and estimation. It also 
describes the methodology used to calculate the 
cost of restoration and the value for replacement 
purposes. The outcome of the claim and future 
plans for collection development at Sussex are 
summarised. The conclusion draws some general 
lessons for future disaster prevention on how to 
minimise risks from flooding and gives advice on 
insuring library collections as well as drawing up 
insurance claims after disasters. 
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Examines the dynamics of knowledge sharing using the 
multi-person game-theoretic framework. Proposes that an 
individual’s knowledge sharing tendency is driven by a set 
of contextualised concerns and interests not unlike the 
notion of payoff in game theory. Furthermore, the 
decision to share or withhold knowledge depends on that 
which yields a higher payoff. With this premise, submits 
two objectives. One is to investigate if an individual's 
perceived payoff of sharing knowledge is contingent on 
the knowledge sharing behaviour of others. The other is 
to analyse the perceived payoff of knowledge sharing and 
determine if it can be characterised by an archetypical 
game in the game-theoretic model. An empirical study 
was conducted among nearly 100 students in a local 
institute af higher education. The scope was confined to 
the students’ willingness to post asynchronous entries to 
an electronic discussion forum. Finds that the individual 
student's perceived payoff of sharing knowledge was 
contingent on the knowledge sharing behaviour of others. 
Furthermore, the perceived payoff o7 knowledge sharing 
among them could be characterised by a multi-person 
assurance game. In conclusion, discusses three 
implications for managers who aim to sustain 
asynchronous knowledge sharing in their organisations. 
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Introduction 


The notion of knowledge sharing has 
attracted much attention from both 
researchers and practitioners in the field of 
knowledge management. Many studies have 
been done to identify the factors that impede 
knowledge sharing. Several best practices 
have also been uncovered in organisations 
that exhibited systematic knowledge sharing 
phenomena. However, there are still gaps in 
our understanding of why and when 
knowledge sharing occurs. Many questions 
related to the dynamics of knowledge sharing 
remain unanswered. For example, why are 
some incentive schemes designed to 
promoting knowledge sharing highly 
effective in some organisations but fail in 
others? Why are there pockets of people who 
passionately share knowledge among 
themselves even though they belong to an 
organisation whose culture resists knowledge 
sharing? When is an individual likely to share 
knowledge? 

For this reason, this paper seeks to advance 
understanding on knowledge sharing by 
carrying out an empirical study that uses the 
game-theoretic framework. It has two 
objectives. One is to investigate if an 
individual’s perceived payoff of sharing 
knowledge is contingent on the knowledge 
sharing behaviour of others. The second is to 
analyse the perceived payoff of knowledge 
sharing and determine if it can be 
characterised by an archetypical game in the 
game-theoretic model. These objectives have 
implications on how managers can promote 
knowledge sharing in their organisations. 

This paper is organised as follows. First, it 
discusses the extant work done on 
knowledge sharing and conceives knowledge 
sharing as a decision made on the basis of a 
contextually determined perceived payoff by 
the one who shares. It further argues that 
such a perspective lends itself suitable for 
analysis using the game-theoretic 
framework. Next, it presents the 
fundamentals of game theory and the three 
archetypical games. Following that, an 
empirical study conducted to test the two 
propositions submitted in this paper is 
described. Finally, this paper analyses the 
findings that emerged from the study and 
discusses three implications for managers 
who aim to sustain asynchronous knowledge 
sharing in their organisations. 
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Knowledge sharing 


Several scholars have sought to distinguish 
between information and knowledge (Kock 
et al., 1997; Davenport and Prusak, 1999; 
Natarajan and Shekhar, 2000). The former is 
commonly defined as data that are being 
interpreted through analysis, correlation and 
summary, while the latter is defined as 
information that is validated and endowed 
with meaning through experiences, beliefs, 
values and insights (Davenport and Prusak, 
1999). Information is usually person- 
independent whereas knowledge is context- 
sensitive (Natarajan and Shekhar, 2000). 
This means that information can be easily 
detached and transferred from its source 
without losing its meaning, but knowledge 
has to be shared within a specific context for 
its essence to be grasped. Since knowledge is 
enriched with insights and is socially 
embedded in organisational practices, it is not 
easily imitated, traded or substituted. Hence, 
knowledge sharing is an important strategy for 
developing a competitive advantage for any 
organisations (McEvily et al., 2000). 
Successful companies are those capable of 
capturing knowledge acquired by one part of 
the organisation and leveraging it for other 
parts. Siemens’ sale team in Malaysia, for 
example, was able to secure a large 
telecommunication contract primarily 
because of the knowledge and experience 
gained by their peers in Denmark (Wenger 
et al., 2002). As a result of introducing 
knowledge sharing initiatives, firms such as 
Buckman Laboratories and Texas 
Instruments have seen enormous gain in 
revenues, while those such as Dow Chemical 
and Chevron have reported tremendous 
savings (O’Dell and Grayson, 1998). 
Knowledge sharing is the process by which 
individuals collectively and iteratively refine a 
thought, an idea or a suggestion in the light of 
experiences. The original idea may be 
progressively modified or gradually rejected 
until a shared perspective emerges (West and 
Meyer, 1997; Rogers, 1986). Knowledge can 
be shared through face-to-face interactions 
(Dixon, 2000), synchronous and 
asynchronous communication using an 
electronic knowledge repository (Lynne, 
2001). The degree of explicitness of 
knowledge shared is influenced by the media 
richness of the communication channels used 
(Chua, 2001). For example, highly tacit 
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knowledge can be shared through face-to-face 
interactions, but knowledge has to be codified 
first when a text-based electronic discussion is 
used. 

As a matter of fact, some extent of 
knowledge sharing takes place among 
organisation members regardless of whether 
this process is deliberately managed. When a 
technician asks a fellow colleague how a work- 
related problem can be solved, knowledge is 
shared when help is given. However, even as a 
natural social phenomenon, unmanaged 
knowledge sharing processes are local and 
fragmentary (Davenport and Prusak, 1999), 
They may neither be efficient nor effective 
(Gupta and Govindarajan, 2000; Szulanski, 
1996), If this is so, the value of knowledge in 
the organisation cannot be fully leveraged. In 
fact, several instances of corporate failures in 
the risk management firms such as Baring 
Securities, Kidder Peabody, and 
Metallgesellschaft Refining and Marketing 
have been attributed to poor internal 
knowledge sharing mechanisms (Marshall 
et al., 1996). For this reason, an important 
agenda of knowledge management initiatives 
is to systematically foster knowledge sharing 
among organisation members (Martin, 2000). 

Many studies have been done to examine 
the factors that impede knowledge sharing. 
One factor is the lack of visible top 
management support for knowledge sharing 
activities (Trussler, 1998). Another factor is 
the failure to make knowledge sharing a basis 
for advancement in the organisation 
(Hiebeler, 1996). Most existing incentive 
systems neither recognise nor reward 
knowledge sharing behaviour. The third 
factor is the ignorance among organisation 
members (O’Dell and Grayson, 1998). 
Especially in large organisations, organisation 
members may not be aware of someone who 
would be interested in the knowledge they 
have or has the knowledge they require. The 
final and probably the most significant factor 
is the lack of a relationship between the 
source and the recipient of knowledge (Chow 
et al., 2000). When there is an absence of a 
strong personal tie that warrants listening to 
or helping each other, knowledge sharing is 
hampered (O’Dell and Grayson, 1998; von 
Krogh, 1998). These factors are mutually 
reinforcing. The resultant effect is a cultural 
resistance to knowledge sharing (Scheraga, 
1998; Ruggles, 1998; Microsoft, 2000; 

De Long and Fahey, 2000). 
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Several strategies that could foster 
knowledge sharing include using technology 
appropriately (O’Dell and Grayson, 1998), 
introducing incentive schemes for knowledge 
sharing (Matusik and Hill, 1998; Trussler, 
1998), cultivating communities of practice 
(Wenger et al., 2002), providing role models 
(Hiebeler, 1996) and creating centres of 
excellence across functional divisions 
(Puccinelli, 1998). A study done on some 40 
companies that exemplify desirable knowledge 
sharing practices revealed that these 
companies not only integrate knowledge 
sharing into their business strategy, they 
develop approaches and styles that match 
their prevailing organisational culture to 
minimise resistance (McDermott and O’Dell, 
2001). For example, Ford openly talks about 
the importance of knowledge sharing, 
conducts internal advertising and organises 
knowledge sharing events. Lotus, on the other 
hand, works through informal teams to share 
knowledge based on their business needs. 
While these recommendations provide useful 
insight in promoting the general climate of 
knowledge sharing in the organisation, they 
provide little explanation on its occurrence. 

There are currently two ways of thinking 
that could shed some light on the underlying 
forces that govern knowledge sharing (Cohen, 
1998). The first is the notion of a knowledge 
market (Davenport and Prusak, 1999). 
Knowledge sharing is conceived as a 
transaction that takes place among buyers, 
sellers and brokers in the knowledge market. 
Given their limited time and energy, 
individuals engage in a knowledge transaction 
only when they believe it will give them a 
worthwhile return on those scarce resources. 
The knowledge market perspective builds on 
the economic exchange theory and it asserts 
that individuals are driven by rational self- 
interest (Gee and Kim, 2002). Knowledge 
sharing will occur when its reward exceeds its 
costs (Monge et al., 1998; Kelley and 
Thibaut, 1978; Constant et al., 1994). The 
second is the notion of knowledge community 
(von Krogh, 1998, Nonaka and Takeuchi, 
1995). This perspective emphasises less on 
knowledge transactions but more on personal 
connection and commitment to shared 
success. On the basis of trust and 
relationships, individuals develop a genuine 
concern for each other. The motivation to 
share knowledge is therefore inspired by love, 
care and generosity. 


Aslib Proceedings 
Volume 55 - Number 3 - 2003 « 117-129 





The knowledge market perspective tends to 
conceive knowledge sharing behaviour as a 
transaction laden with self-serving and 
calculative overtones, while the knowledge 
community perspective tends to conceive 
knowledge sharing behaviour as an activity 
that emerges out of utopia. This paper 
recognises the value of each perspective and 
seeks to converge the two. It proposes that 
knowledge sharing occurrence is governed by 
the perceived payoff, which accommodates 
extrinsic, intrinsic and social dimensions. 
This notion is supported by the theory of 
information sharing (Constant et al., 1994) 
and the two-person game-theoretic 
framework of knowledge transfer (Loebecke 
et al., 1999). 

The theory of information sharing seeks to 
explain the factors that support or constrain 
information sharing. It goes beyond 
communication and information exchanges 
among friends and personal contacts to 
include strangers the sender will never meet. 
The theory considers both organisational 
factors as well as psychological factors such as 
feelings, values, and self-identities (Constant 
et al., 1999). The two-person game-theoretic 
framework of knowledge transfer, on the 
other hand, examines the net value that 
individuals would gain when they share or 
withhold knowledge. The framework 
considers the following components: the basic 
value of each other’s knowledge, the 
synergetic value reaped as a result of sharing, 
the value of monopolistic knowledge which 
can only be retained by withholding, the 
leveragability of sharer’s knowledge from 
which the recipient can gain and the loss of 
valuable knowledge, known also as the 
negative reverse-impact. 

This paper argues that the perceived payoff 
of knowledge sharing has social, economic 
and political bearings specific to the one who 
shares and to the environment in which the 
sharing of knowledge takes place. It 
encapsulates myriad contextual elements 
comprising the interests and concerns, and it 
succinctly represents all that an individual 
really cares about in sharing knowledge in a 
given instance. Such perspective can be used 
to examine why individuals are or are not 
sharing their knowledge, regardless of the 
sophistication of technology, organisational 
mandate, or financial incentives. 

The interests in knowledge sharing 
encompass expected rewards, expected 
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associations and expected contribution (Gee 
and Kim, 2002). These refer respectively to 
extrinsic incentives, enhancement in mutual 
relationships and improvement in the 
organisation’s performance an individual 
expects to reap when knowledge is shared. 
Examples of an individual’s interests in 
sharing knowledge include creating or 
reinforcing of an expert image among peers 
(Jarvenpaa and Staples, 2001), promoting 
one’s visibility in the organisation, fulfilling an 
organisational obligation, establishing future 
relationships, altruistically wanting to help 
others (Davenport and Prusak, 1999), or 
improving the competencies of the 
department. The concerns encompass the 
efforts and costs an individual has to incur to 
share knowledge. Examples of concerns in 
sharing knowledge include the amount of 
time and resources required, the breach of 
confidential knowledge (Weiss, 1999), the 
loss of valuable knowledge, or the risk of 
losing one’s face as a result of sharing 
knowledge that is not regarded to be valuable 
(Chow et al., 2000). 

The relative significance of the various 
interests and concerns of sharing knowledge 
differ from individual to individual and from 
instance to instance. Furthermore, they are 
moulded by national culture (Triandis, 1989; 
Erez and Earley, 1993; Chow et al., 2000), 
organisational culture, personal values and 
self-identities (Jarvenpaa and Staples, 2001). 
In particular, since knowledge sharing is 
heavily social in nature, the perceived payoff 
of knowledge sharing is affected by social 
factors such as trust (Schrader, 1990; 
McDermott and O’Dell, 2001), care (von 
Krogh, 1998), emotional commitment and 
the quality of the relationship (Weiss, 1999}. 
Since social factors are precipitated by a 
bilateral relationship between the one who 
shares and the one who receives knowledge. 
knowledge sharing behaviour can only be 
sustained through reciprocity. An individual 
shares knowledge in reciprocation because it 
is both a socially expected behaviour 
(Constant et al., 1994) as well as a means to 
obtain desired resources for mutual benefits 
(Tyler, 1999). Conversely, when knowledge 
sharing is not reciprocated, knowledge 
sharing behaviour is impeded. This is 
consistent with the social exchange theory 
(Kelley and Thibaut, 1978). When applied to 
a group, this paper posits that an individual’s 
knowledge sharing bekaviour is contingent 
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upon that of others in the group. Hence, the 


. first proposition is submitted: 


P1. An individual’s perceived payoff of 
sharing knowledge in a group is 
contingent on the knowledge sharing 
behaviour of others in the group. 


Game theory and archetypical games 


When knowledge sharing is conceived as a 
decision governed by the perceived payoff, it 
possesses a few distinctive features that are 
also found in the structure of strategic games. 
First, the individuals who engage in 
knowledge sharing are usually defined within 
a context, in an organisation or a community 
of practice. In addition, knowledge sharing 
involves two or more persons. Similarly, 
strategic games are played between two or 
more known players. Next, within the realm 
of knowledge sharing, each individual chooses 
between two decisions, either to share or 
withhold knowledge. Similarly, most strategic 
games involve a set of mutually exclusive 
strategies from which one is chosen. Finally, 
as mentioned earlier, the perceived payoff for 
an individual who contemplates sharing 
knowledge encapsulates all his interests and 
concerns. The decision to either share or 
withhold knowledge is dependent on the 
action that yields a higher payoff. This is 
consistent with the general economic 
principles in everyday life (Friedman, 1997). 
Similarly, in strategic games, the rationality of 
all players is assumed. The goal of every 
player is to decide on a strategy that yields the 
maximum payoff. For the above reasons, 
knowledge sharing is deemed suitable to be 
examined using game theory. 

Having its roots in mathematics and 
economics, game theory is the science of 
rational behaviour in interactive situations 
where the actions of participants alter one 
another’s outcomes (Dixit and Skeath, 1999). 
Essentially, game theory provides a 
framework to analyse the strategic positions of 
the various parties engaged within a scheme 
of choices, each fully aware of the others’ 
objectives, constraints and positions. Game 
theory has been applied to many disciplines, 
including economics, business, political 
science, psychology and philosophy (Dixit 
and Skeath, 1999). It has been used to 
analyse several situations, such as the power 
in the parliamentary assembly, the 
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functioning of auctions, economics of 
information, arbitration procedures, 
bargaining and negotiations between two or 
more parties (Brams, 1990; Ponssard, 1981) 
and knowledge transfers among rival 
organisations (von Hippel, 1994; Schrader, 
1990; Loebecke et al., 1999). 

Two important terms commonly used in 
strategic games are strategies and payoffs. 
Strategies refer to the choices available to the 
players. Payoffs are numerical representations 
associated to each available combination of 
choices of strategies by all the players. The 
payoff structure in a strategic game is unique 
to each player and it denotes the extent to 
which an outcome is favourable for that 
player. The payoffs for one player capture 
everything in the outcomes of the game that 
the player cares about. Hence, it is in the 
interest of the player to choose a strategy that 
yields the maximum payoff. To illustrate the 
strategies available to all players as well as the 
corresponding payoffs, a payoff matrix is 
usually used. An example of a payoff matrix 
can be found in Table I. 

In a strategic game, it is assumed that each 
player chooses the strategy that yields the 
maximum payoff for himself. A dominant 
strategy is one that out-performs all other 
strategies regardless of the choices made by 
the other players. The other strategies are said 
to be dominated strategies. 

For the sake of simplicity, a two-person 
game will first be considered. Each player can 
choose one out of two mutually exclusive 
strategies, generically, to participate or shirk. 
Hence, there are a total of four possible 
decisions jointly made by the two players. 
Assuming that the game is symmetric, the 
payoff of each player will be the same when 
both participate or when both shirk. Based on 
this scenario, the payoff matrix of a two-person 
game can be generalised as shown in Table I. 

Depending on the relative payoff values of a, 
b, c and din Table I, a game can be classified 
as one of the three archetypes, namely, 
prisoners’ dilemma, chicken or assurance 
(Dixit and Skeath, 1995; Kay, 1995). 


Table | Generalised payoff table of a two-person game 


Person 2 
Participate Shirk 
Person 1 Participate a’, a be 
Shirk ê, b f, P 


Notes: “Payoff of Person 1; "Payoff of Person 2 
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Prisoners’ dilemma 

If c >a, a > d, d > b, this game is a prisoners’ 
dilemma. Table II shows an example of a 
prisoners’ dilemma having arbitrary payoffs 
for the players. If Person 2 participates, 
Person 1 receives a payoff of 7 if he/she 
participates and a payoff of 10 if he/she shirks. 
If Person 2 shirks, Person 1 receives a payoff 
of 3 if he participates and a payoff of 5 if he/ 
she shirks. Hence in both cases, Person 1 is 
better off by choosing to shirk. The same logic 
applies for Person 2: he/she is better off by 
choosing to shirk. In a prisoners’ dilemma, 
shirking is therefore the dominant strategy for 
individual players. However, if both players 
choose to shirk, each receives a payoff worse 
off than if they both choose to participate. To 
reap maximum benefit, each player would 
need to co-operate with the other by choosing 
the dominated strategy together. Hence, a 
prisoners’ dilemma is also known as a game of 
co-operation (Kay, 1995). 


Chicken 

If c > a, b > d, then this is a game of chicken. 
Table HI shows an example of a game of 
chicken. If Person 2 participates, Person 1 
receives a payoff of 7 if he/she participates and 
a payoff of 10 if he/she shirks. In this case, 
Person 1 is better off by choosing to shirk. If 
Person 2 shirks, Person 1 receives a payoff of 
5 if he/she participates and a payoff of 3 if he/ 
she shirks. In this case, Person 1 is better off 
by choosing to participate. Hence, in chicken, 
the dominant strategy for the individual 
player does not exist. Instead, the strategy is 
to shirk if the other player participates and 
participate if the other player shirks. Since 
both players maximise their payoffs by not 
choosing the same strategy as the other, 
chicken is also known as a game of 
differentiation (Kay, 1995). 


Table Il Example of a prisoners’ dilemma 


Person 2 
Participate Shirk 
Person 1 Participate 7,7 3, 10 
Shirk 10, 3 5,5 

Table III Example of a chicken 

Person 2 
Participate Shirk 
Person 1 Participate 7,7 5, 10 
Shirk 10, 5 3,3 
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Assurance 

Ifa >c,a>d, d> b, then this is a game of 
assurance. Table IV shows an example of an 
assurance game. If Person 2 participates, 
Person 1 receives a payoff of 10 if he 
participates and a payoff of 7 if he shirks. In 
this case, Person 1 is better off by choosing to 
participate. If Person 2 shirks, Person 1 
receives a payoff of 3 if he participates and a 
payoff of 5 if he shirks. In this case, Person 1 
is better off by choosing to shirk. Hence, in an 
assurance game, the dominant strategy for the 
individual player does not exist. Instead, the 
strategy is to participate if the other player 
participates and shirk if the other player 
shirks. Since both players maximise their 
payoffs by choosing the same strategy as the 
other, the assurance game is also known as the 
game of coordination or Battle of the Sexes 
(Kay, 1995). 


Multi-person games 

In a multi-person game, the payoff for each 
player is dependent on the number of players 
who participate and shirk. Supposing in a 
population of N players, there are n number 
of players who participate. Each participant 
receives a payoff of p(n), while each shirker 
receives a payoff of s(n). 

When a shirker decides to participate, the 
number of players who participate will be + 
1. He would receive a payoff of p(m+1). If he 
decides to remain a shirker, his payoff would 
be s(n). Hence, an individual would 
participate if p(n+1) > s(n) and would shirk if 
p(ntl) < s(n). 

Figure 1 shows the payoff of the multi- 
person prisoners’ dilemma. The vertical axis 
represents an individual’s payoff while the 
horizontal axis represents the number of 
participants in the population. In the 
prisoners’ dilemma, shirking is the dominant 
strategy, regardless of the number of players 
who participate. Hence, s(n) > p(n+1) for all 
values of n (Dixit and Skeath, 1999). When 
the rest of the population participate, an 
individual is better off if he shirks (C > A). 
When none in the population participate, an 
individual is still better off if he shirks (D > B). 


Table IV Example of an assurance game 


Person 2 
Participate Shirk 
Person 1 Participate 10, 10 3,7 
Shirk 7,3 5,5 


Volume 55 - Number 3 - 2003 - 117-129 


Figure 1 Multi-person prisoners’ dilemma 





+ Payoff 











Source: Dixit and Skeath (1999) 





In the game of chicken, p(n+1) > s(n) for 
small values of n, but p(n+1) < s(n) for large 
values of n (Dixit and Skeath, 1999). Figure 2 
shows the payoff of the multi-person chicken. 
When the rest of the population participate, 
an individual is better off if he/she shirks (C > 
A). When none in the population participate, 
an individual is better off if he/she participates 
(D > B). So, in a multi-person chicken, when 
few are participating, the strategy is to 
participate. Conversely, when many are 
participating, the strategy is to shirk. 

In the assurance game, s(n) > p(n+1) for 
small values of n whereas p(n+1) > s(n) for 
large values of n (Dixit and Skeath, 1999). 
Figure 3 shows the payoff of the multi-person 
assurance game. When the rest of the 
population participate, an individual is better 
off if he/she participates (4 > C). When none 
in the population participate, an individual is 
better off if he/she shirks (D > B). So, in 
multi-person assurance game, when few are 
participating, the strategy is to shirk. 
Conversely, when many are participating, the 
strategy is to participate. 

In summary, a multi-person game can be 
illustrated graphically using two lines, namely 
BA and DC that show the payoff of the 


Figure 2 Multi-person chicken 





Source: Dixit and Skeath (1999) 
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Figure 3 Multi-person assurance game 
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Source: Dixit and Skeath (1999) 





individual who participates and shirks 
respectively. At point A and point C, the rest 
of the population participates; at point B and 
point C, none in the population participate. If 
point A > point C and point B > point D, then 
the game is prisoners’ dilemma. If point A < 

point C and point B > point D, then it is a 

game of chicken. Finally, if point A > point C 

and point B < point D, then it is an assurance 

game. 

Argued earlier, when knowledge sharing is 
conceived as a decision made on the basis of 
the perceived payoff, it is compatible with the 
structure of a strategic game. The dynamics 
experienced among a group of known 
individuals contemplating to share knowledge 
is posited to be characterised by a multi- 
person game. The two generic strategies 
described, namely, “participate” and “shirk”, 
are operationalised as “share knowledge” and 
“withhald knowledge” respectively. Hence 
the second proposition is submitted: 

P2. An individual’s perceived payoff in 
sharing knowledge among a group of 
organisation members can be 
characterised by an archetypical multi- 
person game in game theory. 


An empirical study 


Research context 

An empirical study was conducted in a local 
institute of higher education among all 
second-year students (N = 120) in the 
Diploma in Business Information 
Technology. A large majority (> 95 per cent) 
of them were local students whose mean age 
was 18. They were generally well acquainted 
with one another because they had been in the 
same cohort for more than a year. 


Volume 55 « Number 3 - 2003 - 117-129 


Students in the institute had been given 
access to asynchronous electronic discussion 
rooms within which they could interact 
among themselves. Students could post ideas, 
suggestions, questions as well as thread 
responses to the earlier entries posted by other 
classmates. Contributing to the discussion 
room was done entirely on a voluntary basis. 

This study was conducted at the beginning 
of the semester and its scope was confined to 
the students’ willingness to contribute in the 
discussion room designated specifically for an 
information technology module. Knowledge 
sharing was operationally defined as making 
studies-related contributions to the discussion 
room. This included posting answers to 
tutorial questions, sharing a programming tip 
or raising questions about certain topics 
discussed in the lectures. The payoff of 
knowledge sharing was operationally defined 
by a scale that measured the students’ 
perception of the major concerns and 
interests related to making contributions to 
the discussion room. The items on the scale 
were contextually determined from the 
students. 


Research design 
The study was conducted using a two-phase 
data collection method. The first phase 
involved two focus group meetings among ten 
randomly selected students. The purpose of 
the focus group meetings was to identify 
elements that were either of concern or 
interest to the students as they decided 
whether to contribute to the electronic 
discussion room. Hence, an instrument that 
measured the perceived payoff of knowledge 
sharing could be established contextually. 
The second phase involved a questionnaire, 
which was administered to all second-year 
students. The questionnaire was developed 
from the findings elicited from the focus 
group meetings. Its purpose was to measure 
the students’ perceived payoffs of knowledge 
sharing under different situations. For 
parsimonious reasons, only four definitive 
situations have been included: (S1) when 
both the respondent and the rest of his peers 
share knowledge, (S2) when the respondent is 
the only one who shares knowledge but none 
of his peers do, (S3) when the respondent is 
the only one who does not share knowledge 
but all his peers do and (S4) neither the 
respondent nor the rest of his peers shares 
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knowledge. These situations are tabulated in 
Table V. 

P1 can be tested by analysing the variances 
of the payoffs of knowledge sharing under the 
four situations, namely, S1, S2, S3 and S4. S1 
and S2 represent point A and point B on the 
p(nt1) line, while S3 and S4 represent point 
C and point D on the s(n) line in the multi- 
person game payoff graphs shown in Figure 1, 
Figure 2 and Figure 3. P2 can be tested by 
comparing $1 with S3 and S2 with S4 using 
one-tail t-tests. 


Data collection and analysis 

From the two focus group meetings, five 
concerns/interests pertinent to the students as 
they contemplated contributing to the 
electronic discussion room were identified 
using inductive analysis. They were: 

(1) level of understanding of the module; 
(2) final grade of the module; 

(3) self-worth; 

(4) sense of recognition among peers; and 
(5) level of rapport with peers. 


Shown in the Appendix, the questionnaire 
comprises four sections within which these 
five items are presented. Sections 1, 2, 3 and 
4 measure the payoff of knowledge sharing 
under the four situations, namely, S1, S2, S3 
and S4 respectively. 

The response rate to the questionnaire was 
82.5 per cent (n = 99). The reliability of data 
collected was measured by checking for inter- 
item correlations in the variables. Cronbach’s 
alpha was used because it provides a good 
estimate of reliability and is preferred over the 
split-half method (Nunnally, 1978). Table VI 


shows Cronbach’s alpha coefficients of the 
Table V Four situations in knowledge sharing 
The individual 
Shares Does not 
knowledge share 
The rest of Share knowledge S1 $3 
his peers Do not share 82 S4 


Table VI Cronbach's alpha coefficients of variables in 
the questionnaire 





Variables Cronbach's alpha coefficient (n = 99) 
S1 0.7366 
$2 0.7512 
53 0.7428 
S4 0.8584 
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four variables in the questionnaire. Since 
Nunnally (1978) suggested the value of at 
least 0.70 as the basis of reliability for 
Cronbach’s alpha coefficient, all responses 
collected had passed the reliability test. 

Table VII shows the mean scores and 
standard deviation obtained for each variable. 

Figure 4 shows the values of S1, $2, S3 and 
S4 plotted on the payoff graph. A linear 
relationship between the payoff and the 
number of participants is assumed. 

Since the data were interval-scaled and the 
sample size was more than 30, normal 
distribution could be assumed. Furthermore, 
the standard deviations were also known. 
Hence, it would be appropriate to use 
ANOVA to test P/ and a one-tail t-test to test 
P2. 

At a 5 per cent level of significance, $1, S2, 
S3 and S4 were found not to have equal 
variances (p < 0.05). This means that a 
student’s perceived payoff of sharing 
knowledge varied according to the different 
situations which depict whether he and the 
rest of his peers share knowledge or otherwise. 
P1 is thus supported. 

At a 5 per cent level of significance, S1 > S3 
(p < 0.05) and S2 < S4 (p < 0.05). The first 
inequality means that when the rest of his 
peers share knowledge, a student is better off 
sharing knowledge than if he did not. The 
second inequality means that when the rest of 
his peers do not share knowledge, a student is 
better off not sharing than if he did. Such a 


Table VII Descriptive statistics of $1, $2, 53 and $4 


Variables Mean score Standard deviation 
$1 3.79 0.54 
$2 2.47 0.67 
$3 2.74 0.78 
S4 2.73 0.62 


Figure 4 Values of $1, $2, $3 and $4 plotted 
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payoff structure corresponds to that of a 
multi-person assurance game. P2 is thus 
supported. 


Discussion 


Three findings have emerged from this study. 
First, different groups of individuals in 
different contexts hold different interests and 
concerns about knowledge sharing. In this 
study, the participants, who were students, 
had some five main interests and concerns 
that were mostly related to cognitive 
performance and relational issues. If this 
study were to be replicated in a typically fast- 
paced organisation in which people juggle 
among different priorities, such as career 
aspiration, family commitment and meeting 
tight project deadlines, the interests and 
concerns associated with knowledge sharing 
would greatly differ from that found in this 
study. For example, the concern about 
expending time to share knowledge, which 
was not raised by the participants in this 
study, might have been elicited. Furthermore, 
other interests and concerns like promotion 
prospects and building an expert image might 
also have been included. 

It is worthwhile for managers responsible 
for engendering knowledge sharing in their 
organisations to first understand the issues 
that either propel or hinder knowledge 
sharing tendency among members in the 
group. This is possible only when managers 
are well acquainted with the group and are 
knowledgeable about the members’ interests 
and concerns in sharing knowledge. 
Managers can choose from a range of data 
collection methods such as focus group 
discussions, surveys, one-to-one talks or off- 
site meetings to elicit the issues pertinent to 
the members. The issues may include a host 
of motivational factors, de-motivational 
factors, personal aspirations, work priorities 
and the impact of practices enforced 
organisation-wide, for example, as a result of 
the intention to achieve ISO certification. 
From among these issues, managers need to 
identify and prioritise the more significant 
ones for follow-up actions. 

Thereafter, managers can introduce 
interventions that specifically address the 
significant issues. For example, if members 
are concerned about not receiving enough 
management recognition, then overt 
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acknowledgement from the management 
could be given as an incentive for 
participation. In addition, salient points and 
the names of the contributors can be culled 
from the discussions periodically and be 
submitted to the management for their 
attention. If members are concerned about 
the difficulty of contributing an entry into the 
discussion forum, then the user-friendliness 
of the system has to be dealt with. 
Furthermore, user-training sessions, helpdesk 
support and other technical assistance need to 
be put in place. 

Second, since S1, $2, S3 and S4 were 
found not to have equal variances, this implies 
that an individual’s perceived payoff varied 
according to the joint decision between 
himself and the rest of his peers to share 
knowledge or otherwise. The notion of payoff 
represents an aggregation of all interests and 
concerns held by an individual and its 
purpose is to serve as proxy to an individual’s 
likelihood to share knowledge. Hence, the 
confirmation of P1 means that an individual’s 
decision to share knowledge is influenced in 
part by the decisions of others to share 
knowledge as well. 

It must be recognised that knowledge 
sharing does not occur in a social vacuum but 
is engendered through the reciprocal 
relationships among members in the group. 
As generosity begets generosity, miserliness 
begets miserliness. Thus, managers who wish 
to promote asynchronous knowledge sharing 
need to establish norms of cooperation, 
cordiality, goodwill and trust in the group. 
These efforts require managers to foster 
interaction within the group not merely 
through the electronic platform, but also face- 
to-face encounters. For example, managers 
can encourage informal chat around the 
water-cooler, organise holiday parties (von 
Krogh, 1998), deploy staff to be involved in 
taskforce and cross-functional teams (Nonaka 
and Konno, 1998). 

Third, an individual’s perceived payoff of 
knowledge sharing varied with the number of 
people who participated in knowledge 
sharing. Given the number of people who 
participated in knowledge sharing, an 
individual’s perceived payoff of knowledge 
sharing and that of withholding knowledge 
differed. In particular, when the rest of his 
peers share knowledge, an individual is better 
off sharing than withholding knowledge (S1 > 
S3). When the rest of his peers withhold 
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knowledge, an individual is better off 
withholding than sharing knowledge (S2 < 
$4). Such payoff structure matches with that 
of a multi-person assurance game. This not 
only confirms P2, it also means that the 
principle of reciprocity was found to operate 
among the participants in this study. 

When a linear relaticnship between the 
payoff and the number of participants is 
assumed as shown in Figure 4, an individual’s 
perceived payoff to share knowledge 
intersects with that to withhold knowledge at 
the point where the number of participants 
who shared knowledge is 20 (20.2 per cent). 
In other words, an ind-vidual was better off 
sharing than withholding knowledge when 
there were at least 20 participants who shared 
knowledge. However, given the small 
population size targeted and the specific 
context in which this scudy was done, this 
finding has to be treated with caution. 
Moreover, in reality, S1-S2 and S3-S4 in 
Figure 4 do not have to be straight lines and 
may intersect at more than one point. Hence, 
the minimum participation rate of 20.2 per 
cent of the population needed to create the 
propensity for knowledge sharing may not be 
a generalisable figure. Even so, the results still 
have practical implications for managers. For 
knowledge sharing behaviour to be 
engendered and sustained effectively among 
participants accessing a common electronic 
repository, participation rate has to exceed a 
minimum threshold. When that happens, a 
positive reinforcement of knowledge sharing 
will be invoked. The challenge for managers 
who are responsible for launching electronic 
discussion forums for a group of people in the 
organisation is therefore to ensure as high a 
participation rate as passible at the initial 
stage. This would generate the impetus to 
keep the contributions forthcoming. 
Suggestions to foster a high initial 
participation rate include intensive internal 
marketing, using incentives such as a lucky 
draw, identifying a theme that is of great 
interest or concern to the group and seeking 
to solicit members’ views via the electronic 
forum. 


Conclusion 


This paper represents a first step towards 
advancing the understending of knowledge 
sharing using a multi-person game-theoretic 
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framework. A major argument submitted is 
that knowledge sharing is governed by a 
perceived payoff contextually specific to the 
one who shares and to the environment in 
which the sharing takes place. The perceived 
payoff encapsulates all the interests and 
concerns pertinent to that individual. This 
perspective helps explain why some incentive 
schemes are effective in some organisations 
but fail in others. It also sheds light on 
knowledge sharing occurrences among a 
group of people. 

The empirical work was done among 
students in an institute of higher education 
and the notion of knowledge sharing was 
confined to the posting of entries in an 
electronic discussion forum. From the data 
collected, it was confirmed that an 
individual’s likelihood to share knowledge 
was contingent on the likelihood of others to 
do likewise. In addition, the perceived payoffs 
of knowledge sharing among a group of 
individuals could be characterised by a multi- 
person assurance game. 

The findings which have emerged from this 
study have the following implications for 
managers who aim to sustain asynchronous 
knowledge sharing among a group of people 
in their organisations. First, managers need to 
understand the interests and concerns of the 
people and introduce specific interventions to 
improve the perceived payoff of knowledge 
sharing. Second, managers need to prioritise 
face-to-face interaction among the people and 
establish the norms of cooperation. Third, in 
launching the electronic forum for knowledge 
sharing, managers need to target an initial 
minimum participation rate to create the 
propensity for more contributions. 
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Appendix. Questionnaire — Perceived payoffs of sharing knowledge 


The purpose of this survey is to collect opinions on how you think you will be affected by the 
various joint decisions between yourself and the rest of your peers with regard to contributing 
knowledge to the electronic discussion. There are no right and wrong answers. Please respond 
to every question in this survey. 


Table Al 


Section 1: Assuming that BOTH you and all your peers contribute knowledge, ideas, questions and answers to the 
electronic discussion board 





1 How do you think this situation will affect your level of understanding of the module? 
(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot ol 
2 How do you think this situation will affect your final grade of the module? 
(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot g 
3 How do you think this situation will affect your self-worth? 
(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot g 
4 How do you think this situation will affect your sense of recognition among your peers? ~ 
(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot g 
5 How do you think this situation will affect your level of rapport with your peers? 
(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot oO 


Section 2: Assuming that you are the only one who contributes knowledge, ideas, questions and answers to the 
electronic discussion board but NONE of your peers does 





6 How do you think this situation will affect your level of understanding of the module? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot O 
7 How do you think this situation will affect your final grade of the module? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot O 
8 How do you think this situation will affect your self-worth? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot oO 
9 How do you think this situation will affect your sense of recognition among your peers? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot oO 
10 How do you think this situation will affect your level of rapport with your peers? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot oO 


Section 3: Assuming that ALL your peers contribute knowledge, ideas, questions and answers to the electronic discussion 
board but you DO NOT 


11 How do you think this situation will affect your level of understanding of the module? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot o 
12 How do you think this situation will affect your final grade of the module? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot oO 
13 How do you think this situation will affect your self-worth? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot g 
14 How do you think this situation will affect your sense of recognition among your peers? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot O 
15 How do you think this situation will affect your level of rapport with your peers? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5} Gain a lot g 





(continued) 
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Table Al 


Section 4: Assuming that NEITHER you nor any of your peers contribute knowledge, ideas, questions and answers to the 
electronic discussion board 











16 How do you think this situation will affect your level of understanding of the module? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot O 
17 How do you think this situation will affect your final grade of the module? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot g 
18 How do you think this situation will affect your self-worth? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot oO 
19 How do you think this situation will affect your sense of recognition among your peers? 

(1) Lose a lot (2) Lose some (3) Neutral (4) Gain some (5) Gain a lot o 
20 How do you think this situation will affect your level of rapport with your peers? 

(1) Lose a lot {2} Lose some (3) Neutral (4) Gain some (5) Gain a lot Oo 

Thank you 
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Introduction 


In 1995 the law firm of Baker & McKenzie 
announced that its Chicago office was 
terminating the contracts of many of its 
support staff and was outsourcing its entire 
law library services. Baker & McKenzie was 
engaged in a radical restructuring or re- 
engineering of key business processes and 
resources were reallocated in the light of new 
operational objectives. 

The news shocked information 
professionals in many parts of the world. 
Outsourcing the library operations to 
competing information intermediaries 
challenged the previously undisputed role of 
in-house librarians to meet the practitioners’ 
information requirements. In addition, their 
decision suggested that the consultants and 
decision makers in this particular situation 
were either ignorant of the substantial 
literature on the value of information and 
information services to an organisation, or 
they disregarded it. 

The reconstruction at Baker & McKenzie 
was not an isolated incident and closures, . 
downsizing and outsourcing of corporate 
information services continued into the new 
century (Merrill, 2001). Professional 
responses to this situation range from 
marketing the potential value of library and 
information services to measuring actual 
performance by way of increasingly 
sophisticated metrics. This article reports on 
an approach that incorporates recognised 
business practices to the evaluation of 
corporate library and information servicés. 


Study parameters and strategy 


The authors of this report were concerned 
with the determination and demonstration of 
the value of library and information services 
in a South African law firm at the beginning of 
the twenty-first century. An investigation of 
service evaluation suggested the following 
research questions: 
* How did the nature of the corporate 
entity influence the determination of the 
value of its information support services? 
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«e To what extent did existing performance 
models and standards offer useful guides 
to measuring value? 

* To what extent could financial 
accountability contribute to the 
perception of the value of corporate 
information services? 

* How did the information professional 
determine that the information services 
met the information needs of users? 

+ Was it possible to determine return on 
corporate investment with regard to 
library and information services? 

¢ What controls were in place to ensure 
that operational procedures met the 
required operational standard? 


The authors assumed that the answers to the 
questions above would indicate the 
perception of the value of information services 
in South African law firms and would suggest 
strategies to demonstrate this value to 
corporate management. In addition, they 
expected that the findings of these 
investigations might indicate if South African 
law firms were vulnerable to outsourcing or to 
closure. 

The research study expanded into a series 
of studies that analysed and reported various 
attempts to demonstrate the value of library 
and information services to corporate 
leadership. Findings were based on 


observations of certain operating procedures . 


in a Cape Town law firm seen in relation to 
those in comparable organisations. 
Colleagues in South Africa were approached 
individually and collectively for assistance by 
way of surveys, questionnaires and electronic 
mail communications. The leadership of the 
South African Organisation of Law Libraries 
(OSALL) agreed to permit a number of 
online surveys to be held via the 
organisation’s online listserv in 2001 in order 
to assist in this study. Individual responses 
were submitted off the list so that 
confidentiality could be maintained, but 
summaries were published so that subscribers 
to the listserv had access to the results of the 
surveys. 

Some of the findings were presented at the 
Law Librarian Symposium organised by the 
Institute for International Research in 
Johannesburg, South Africa in 
November 2001. 
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Findings 


Corporate entity and determination of 
value 

Core business processes with regard to the 
incorporated law firm in South Africa were 
identified with the provision of legislative and 
judicial material for legal practitioners. The 
function of the information services in this 
environment was to create reliable systems 
that provided access to appropriate and 
relevant information sources. The systems 
were judged according to the ease and 
effectiveness with which the end-user gained 
access to the information resources in the 
organisation and the assurance of the 
accuracy of these resources. Performance 
objectives were also identified in relation to 
accuracy and response time. It was seen that 
the information resources of the private law 
firm library had to be available and accessible 
to information users at all hours without the 
assistance of the information professional. 
This had implications both with regard to the 
organisation of resources and the provision of 
training of information users. 

The fact that users of information in the 
incorporated legal practice were also its 
shareholders was seen to impact on certain 
management and operational objectives. 
Multiple constituency models that were seen 
to be relevant in evaluating services where 
user requirements were significantly different 
from management requirements had less 
application in the private or incorporated 
environment. User satisfaction could not be 
separated from corporate identity and the 
information professional was expected to be 
accountable with regard to both service 
quality and financial responsibility. 


Evaluation and performance models 
Performance literature suggested that existing 
performance models and standards offered 
useful guides to measuring value. In this 
research study modelling was seen to be 
helpful in integrating principal business 
processes at a conceptual level. Its usefulness 
as a framework for evaluating library and 
information services at operational level was, 
however, doubtful. The introduction of 
performance indicators drew attention to the 
relationship between measures of input cost, 
for example, and the use made of a product or 
service. This had implications for measuring 
operational efficiency and effectiveness, but 
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there was no evidence to suggest that the 
measurement of performance against defined 
performance indicators was sufficient to 
determine or demonstrate the value of the 
services to corporate management. 


Financial accountability and the 
perception of value 

The consideration of the extent to which 
financial accountability contributed to the 
perception of the value of corporate 
information services constituted a significant 
study of cost accounting. An empirical study, 
known to participating organisations as the 
Infospend study, was conducted in seven 
South African law firms during the financial 
year 2000/2001. Participation was by open 
invitation on the online listserv of OSALL. 


Objectives of the Infospend study 

The primary objectives of the exercise were: 

* to identify the information resources that 
were required to support the information 
services provided to legal practitioners in 
a South African private law firm; 

* to capture data relating to the cost to the 
law firm of provid:ng these information 
services during the course of the financial 
year; 

* to investigate whether there was any 
correlation in patterns of information 
spending in South African law firms; and 

* to test whether applying a cost per 
practitioner calculation to data from 
organisations of differing size would yield 
comparable data. 


The secondary objectives were: 

* to analyse relative spending on print- 
based and electronic media; 

* to determine the cost of the labour 
component of the information services in 
an organisation; 

* to determine whether cost per 
practitioner data could be applied to drive 
a cost benefit analysis, or alternatively 
drive an operational value study; and 

* to contribute towards a culture of data 
sharing that might result in benchmarking 
certain operations and developing 
minimum operational standards and best 
practices to guide :nformation 
professionals in South African law firm 
libraries and information centres. 


The cost per practitioner indicator was 
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organisations of differing size. User was 
defined as each income-generating 
practitioner — including candidate attorneys — 
for whom information services were provided; 
and cost was understood to encompass the 
cost to the organisation of information 
resources for practitioners in supporting the 
work of fee earners. Costs included net price 
of all information expenses for which the 
organisation was invoiced and gross amount 
paid to information staff, including benefits. 
Infospend specifically excluded certain costs 
associated with supplying and maintaining 
information technology services available to 
the organisation as a whole. Excluded also 
was the shared cost of maintaining the general 
premises with regard to rates, rental, lighting, 
cleaning services, catering, and administrative 
support. 


Results of the Infospend study 
The findings of Infospend were 
unspectacular. The number of participants 
(seven) was too small a sample for the data to 
be able to express fully the South African 
situation. There was very little correlation in 
the data and nothing against which to 
measure the validity of the values. It was 
expected that economy of scale would 
disadvantage the smaller organisations. It was 
also anticipated that there would be distinct 
differences between organisational allocation 
of resources on information services in line 
with cultural or political differences. Both 
these factors would have impacted on the 
results, but with a small sample there was no 
way of determining which of the two was the 
more significant. Participation in the study 
was voluntary and data related to 
organisations where information services were 
well managed and monitored. The results 
were seen, therefore, to be both incomplete 
and skewed towards technically and 
professionally supported information services. 
The core application of the Infospend ratio 
was the determination of the relative cost of 
maintaining an information service. It was 
accepted that inflationary pressure would 
render the figures out of date and therefore 
irrelevant beyond the year of study. The 
description of data in terms of ratios and 
percentages was therefore seen to be a more 
useful guide to relative spending. 
Print-based resources remained the 
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(see Figure 1). Even when material was 
available digitally and was accessed for its 
sophisticated search capabilities, many 
practitioners preferred the flexibility of 
printed texts when they were required to skim 
large sections of text, or when they wished to 
take texts into consultations. Expenditure on 
print-based resources ranged from 37 per cent 
to 53 per cent of the total amount spent on 
information services in the sample group for 
the financial year 2000/2001, with the median 
at 40 per cent and the average 42 per cent. 

Expenditure on electronic information 
sources ranged from 4 per cent to 20 per cent 
of the total amount spent on information 
services in the sample group. Of the seven 
organisations that supplied data, five spent less 
than 10 per cent of their annual information 
service budget on electronic subscriptions. 

Library and information services that were 
outsourced in many South African law firms 
during the period included loose-leaf 
updating, book cataloguing, document 
delivery, current awareness and alerting 
services, and legal and general research. 

The amount allocated to sundry items like 
the cost of binding journals and the cost of 
special stationery and library items was 
generally low. The Infospend participants 
were not asked to submit data relating to 
general overhead expenses. 

Salary was regarded as the cost to the 
company of employing its information 
professionals and included contributions to 
medical aid, pension or provident fund and 
any other benefits. It also included conference 
fees and membership fees of professional 
organisations. It was sometimes more difficult 
to obtain figures relating to the cash value of 
benefits than it was to obtain salary figures. 
Participation in group benefit schemes, for 
example, was not always reflected on a salary 
advice notification and precise details had to 


Figure 1 Average relative spending 2000/2001 
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be obtained from human resources 
management. Expenditure on salaries and 
benefits of information professionals ranged 
from 31 per cent to 54 per cent of total 
information expenditure in the organisations 
studied. 

Across the seven participating organisations 
the average cost to the law firm of supplying 
library and information services to its legal 
practitioners in private practice was 
ZAR7,324.57 (about USD950) per 
practitioner for the financial year 2000/2001. 


Application of the financial 
accountability exercise 
Infospend tested a method of data capture 
across organisations of varying size. This 
provided a platform from which comparative 
studies and benchmarking was made possible. 
Infospend also provided an aggregation of 
core data against which to measure current 
and projected future spending patterns. This 
vested greater control over financial records in 
the hands of the information professional and 
promoted accountability. Improved capability 
to establish relationships between types of 
information expenses could be used, for 
example, to determine the relative 
expenditure on print-based and electronic 
resources, or to track departmental spending, 
or to position labour costs in relation to the 
overall information expenditure. 
Determining the cost of maintaining a 
corporate information service was seen to be a 
useful point of departure for other studies in 
service evaluation. Follow-up studies that 
were indicated were: 
* an investigation of cost effectiveness of 
the service; 
« a detailed analysis of the exploitation of a 
key resource, staff time; and 
* a consideration of cost recovery as a 
visible sign of added value. 


Determination of the information needs 
of users 

One of the studies that followed from the 
Infospend exercise investigated the 
perception that cost effectiveness could be 
measured in relation to service penetration. 
This was seen to relate directly to the 
assumption that evaluation of services must 
take into account the information needs of its 
users. The authors devised an operational 
value study in an attempt to ascertain use and 
usefulness of information resources and 
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services to the customer. For a period of one 
month all information and legal practitioners 
in a Cape Town law firm were asked to log 
their use of information resources and 
services. Practitioners were given the 
opportunity to identify problems and 
deficienciés with regard to information 
resources in special areas of practice, and to 
articulate the degree of satisfaction with 
information services. 

The results were inconclusive. The actual 
use that was made of information resources 
and services during the given period did not 
appear to correlate with the perceived value of 
the resources that was articulated during 
interviews. The significance of the study was 
seen primarily in its success as a marketing 
exercise; the outcome a commitment to 
customisation of information services in an 
effort to develop the perceived value of the 
services to the individual information users. 


Return on corporate investment with 
regard to library and information 
services 

The authors considered the possibility of 
determining return on capital employed with 
regard to library and information services. 
Claims in the literature for financial return on 
investment suggested that information 
services could become profit generating. 
These claims were explored in the context cf 
the South African law firm with regard to the 
practicality of monitoring those information 
services where costs could be recovered. It 
was assumed that this was a practice that 
would clearly demonstrate the value of library 
and information services to corporate 
management. Cost recovery was associated 
with the cost of labour to perform various 
services and operations and transaction 
details were noted as the time taken over the 
activity was logged. 

There was no problem with the recording 
or analysing of data in order to determine the 
actual cost of library and information services. 
What was shown to be problematic was the 
implementation of cost recovery practices in 
the context of corporate library and 
information services. Legal and ethical 
difficulties were revealed with regard to the 
reselling of information that was copyright 
protected, for example. There was also 
evidence from colleagues to suggest that cost 
recovery and generation of profit through the 
sale of information services might have a 
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detrimental effect on perceptions of the role 
of corporate services. Situations arose where 
inter-departmental charging for the use of 
corporate library and information services led 
to a dramatic decline in the use that was made 
of these services (Toth, 2001). 


Controls and operational standards 

The final question with which this 
investigation of evaluation was concerned was 
the provision of control mechanisms to ensure 
that operational procedures met the required 
operational standards. In terms of South 
African statutory law it was seen that 
incorporated organisations were obliged to 
prepare a set of audited financial records for 
shareholders. However, while legal practices 
were subject to an annual external audit, there 
was no evidence that support services like 
library and information services were 
expected to supply unit reports for the 
company audit or even to conform to certain 
minimum standards. 

The findings with regard to financial 
accountability as a result of the Infospend 
study suggested that other areas of 
operational activity should be subject to 
internal scrutiny. It was suspected that 
information professionals in private law firms 
were unfamiliar with formal performance 
measurement and this was tested in an 
informal electronic survey conducted on the 
OSALL listserv in November 2000. 
Information professionals at 26 organisations 
were sent a short questionnaire by e-mail in 
which they were asked to comment on some 
aspects of performance measurement and 
internal control procedures. The results 
appeared to confirm the perception that 
information services in South African law 
firms were not generally subject to formal 
evaluation procedures. In the apparent 
absence of formal performance standards and 
mechanisms to demonstrate accountability, 
the researchers considered two international 
practices for the determination of value. 
These were accreditation in line with the 
International Organization for 
Standardization’s ISO 9000 standard, and the 
practice of internal auditing. The systemic 
approach exhibited in both these practices 
suggested the merit of developing a similar 
procedure for the evaluation of corporate 
library and information services, an 
information services audit. 
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ISO 9000: quality management 

The ISO 9000 suite was published in 1987 as 
a guide to quality management systems. This 
was the first ISO standard to be concerned 
with process rather than with product (ISO 
Online, 2001). It was understood to be the 
means by which an organisation enhanced 
customer satisfaction by meeting customer 
demands while at the same time satisfying 
regulatory requirements. Compliance with 
the standards developed as part of the ISO 
9000 suite indicated a commitment on the 
part of an organisation to conform to certain 
operating practices. It also indicated the 
corporate intention to commit itself to 
continual improvement in quality 
management. The ISO standard ISO 9001: 
2000 6.1 identified both the resources needed 
to support the quality system as well as those 
resources needed to improve it (Praxiom 
Research Group, 2000). In December 2000 
the system standards in ISO 9001, 9002 and 
9003 were merged into a single requirements 
standard ISO 9001: 2000 (ISO Online, 
2001). 


Internal auditing 


Internal auditing is an independent, objective 
assurance and consulting activity designed to 
add value and improve an organization’s 
operations. It helps an organization accomplish 
its objectives by bringing a systematic, 
disciplined approach to evaluate and improve 
the effectiveness of risk management, control, 
and governance processes (IIA, 2000). 


The internal audit was seen to be a 
mechanism for evaluating internal systems. Its 
scope could be defined in relation to quality 
of performance and management 
effectiveness (Barlow et al., 1995). While the 
internal audit was acknowledged to be a 
voluntarily exercise, the practice of internal 
auditing was governed by the Institute of 
Internal Auditors (IIA, 2000) and local 
affiliates in most countries. The significance 
of the internal audit was seen to lie in the 
objective analysis of procedure and 
performance, where an independent examiner 
(ideally) verified the accuracy of documentary 
evidence. 


Information services audit 

The proposed information services audit is 
distinguished from the information audit 
associated with the work of earlier authorities. 
Burk and Horton (1988) approached the 
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exercise from the perspective of inventory. 
Their methodology identified and mapped 
information resources in an establishment. 
Orna’s (1990) information flow analysis 
extended the scope of inventory to account 
for those processes where value was added 
and information created. Buchanan and Gibb 
(1998) devised a strategy to co-ordinate these 
diverse approaches. 

The information audit is typically 
undertaken in order to assess the value and 
potential value of corporate information to an 
organisation. The information services audit, 
on the other hand, takes its cue from internal 
auditing practices and approaches the analysis 
from the more uncompromising perspective 
of the corporate balance sheet. It is suggested 
that there is a strong case for the 
incorporation of internal auditing principles 
and practices in the traditional reporting 
structure within the organisation. It is 
proposed that the information services audit 
embodies the search for a methodology to 
evaluate information services. It is 
furthermore claimed that the information 
services audit embodies recognition of 
accountability as the critical factor in the 
demonstration of the value of corporate 
information services. 

This paper concludes with an illustration of 
how library and information services in a law 
firm might be systematically prepared to 
conduct an annual information services audit. 
This framework is an adaptation of the ten- 
step management process devised by auditors 
with Deloitte and Touche to assist their 
clients to prepare for internal auditing. The 
process was identified as follows (Barlow et al., 
1995): 

* Step 1. Understand the unit’s business. 

* Step 2. Establish what needs the unit is or 
should be satisfying. 

+ Step 3. Define the unit’s mission. 

+ Step 4. Identify the key activities required 
to achieve the mission. 

» Step 5. Establish performance objectives 
for these key activities. 

+ Step 6. Assess the risk relating to these 
performance objectives. 

* Step 7. Decide what control strategies to 


pursue. 

* Step 8. Implement appropriate control 
strategies. 

* Step 9. Monitor and evaluate actual 
performance. 


+ Step 10. Review performance objectives. 
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An approach to the information services 
audit in a corporate law library 


Understand the unit’s business 

The business of the information services unit 
in an incorporated law firm is identified as the 
provision of legislative and judicial 
information to support the work of the 
practitioner. 


Establish what needs the unit is or should 
be satisfying 

The business of establishing what needs the 
unit should be satisfying is viewed both with 
regard to its users and in conjunction with 
fiscal responsibility to the other stakeholder in 
the unit’s operations, its shareholders. Since 
shareholders in an incorporated law firm are 
also its practitioners there should be no 
conflict of interest in satisfying the 
requirements of both stakeholder groups. 


Define the unit’s mission 

The mission of the information services in an 
incorporated law firm is seen to be derived 
from an understanding of the unit’s business 
and the core production processes in relation 
to stakeholder interests. The mission is vested 
in the creation and maintenance of reliable 
systems to provide access to accurate and 
relevant legislative and judicial information 
resources. With regard to financial 
accountability, the mission is to manage the 
allocation of money ard manpower resources. 


Identify the key activities required to 
achieve the mission 

Key activities are identified as those that 
support stakeholder interests with regard to 
legal information for practitioners and also 
with regard to optimising the corporate 
resources. The key activity that provides for 
the needs of shareholders is the maintenance 
of financial records and supporting 
documentation to account for the financial 
investment of the organisation in its library 
and information services. 


Establish performance objectives for 
these key activities 

Performance objectives with regard to the 
maintenance of sound financial records 
include the methodical recording of all 
expenses and practical arrangements for 
accounts to be settled punctually. An example 
of a performance objective might be that 
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invoices are processed for payment on the 
same day as they are received. This would be 
regarded as the optimal standard and 
performance might be measured with regard 
to the number of times the standard is met 
within a given period. Another performance 
objective might be the generation of financial 
reports within one week of the end of the 
financial year. 


Assess the risk relating to these 
performance objectives 
Risk is seen to operate in all areas where 
process is critical and where failure to adhere 
to performance standards might be 
detrimental to corporate operations. 
Unsatisfactory response time with regard to 
payments to suppliers can, for example, 
compromise the reputation of the information 
professional and the organisation. Failure to 
keep accurate financial records might result in 
ignorance of the financial state of affairs and 
effectively undermine efforts to detect and 
rectify problems. 

Correctly interpreted, service complaints 
are seen to hold important lessons for analysis 
and management of risk. 


Decide what control strategies to pursue 
Control strategies centre on developing 
service quality assurance such that 
performance objectives can be achieved. 
Assurance is seen in terms of effectiveness, 
operational efficiency, reliability of financial 
reporting, and compliance with applicable 
laws and regulations. Assurance is further 
identified as the capability for a system to 
function efficiently and effectively if the work 
passes into the hands of a substitute operator. 
ISO 9001:2000.4 identifies the stages in the 
development of systemic requirements for 
quality assurance in terms of the following: 
* establishment and description of the 
operational processes; 
* documentation of procedures; and 
* maintenance of records. 


Risk with regard to delivery and proper record 
keeping is managed with the introduction of 
procedural checklists and with scrupulous 
attention to cost accounting and reporting. 
The aims of procedural and reporting control 
strategies are to achieve optimal accuracy in 
the systemic and systematic organisation of 
information resources and to make provision 
for substitute service operators. Follow-up 


136 


Accountability and demonstration of the value of information services 


Aslib Proceedings 





Olwyn Garratt and Adeline du Toit 


procedures might, for example, be developed 
to ensure that accounts submitted for 
payment are paid within a prescribed period 
and that acknowledgement of payment is 
recorded. 


Implement appropriate control 
strategies 

Implementation is a management function by 
which proposed control strategies are made 
operational. Procedures are devised to 
accommodate customer expectations and 
define managerial responsibilities both within 
the unit and inter-departmentally. 


Monitor and evaluate actual 
performance 

Objective methods of evaluation involve the 
analysis of documentary evidence that 
measures operational activities against the 
standard of optimal performance. Auditing 
typically centres on the inspection of 
documents that validate the annual financial 
statements. An information services audit 
must take cognisance of the requirement for 
documentary evidence of performance 
success and failure. 


Review performance objectives 
Performance objectives are subject to review 
in the light of operational results. Where 
optimal performance (100 per cent delivery) 
is judged to be achievable — the processing of 
invoices on the day of receipt, for example — 
the relevant performance objectives 
incorporate this standard into management 
practice. Where optimal performance is not 
achievable, the objectives and procedures are 
reviewed to incorporate realistic goals. 


Conclusion 


An investigation of the evaluation of library 
and information services in a South African 
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law firm indicated that there was a need to 
demonstrate operational excellence to 
corporate management. This further 
indicated that evaluation of support services 
in a business environment should be 
considered in the light of corporate business 
measures of accountability, specifically 
quality management and internal auditing. 

An information services audit embodying 
the business practice of internal auditing in 
the light of the demands of quality 
management systems was proposed as a 
statement of accountability and a 
demonstration of the value of information 
services in South African law firms. 
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Abstract ` 


Provides a summary of a Department of Health funded 
research study investigating performance and impact of 
four pilot digital interactive television services in the 
consumer health field. These were launched in various 
locations in the UK in 2001. Text and video, interactive 
and transactional services were featured. Pilots were 
investigated using a combination of research methods. 
The four pilots’ performance varied, but overall there was 
sufficient evidence to suggest that consumer health 
digital interactive television has a healthy future. 
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Introduction 


In 2001, the Department of Health (DoH) 
launched four pilot projects to explore the 
feasibility and effectiveness of health 
information and advice services supplied to 
the public via digital interactive television 
(DiTV). At the same time, the DoH 
commissioned a research team comprising 
academics from City University and the 
University of Sheffield to undertake a 
substantial evaluation of the performance of 
the pilot DiTV operators. This research 
incorporated a suite of methodologies 
through which a variety of types of data were 
collected. 

The four DiTV pilots offered distinctive 
services. Although there were some 
overlapping features among these services, 
they had many more qualities that are 
distinctive. These qualities included the type 
of platform on which a service was 
transmitted, the amount and nature of 
content, the presentation formats used, and 
the degree of interactivity involved in each 
case. The four consortia were: 
Communicopia, Flextech Telewest — Living 
Health, Channel Health and DETV (A 
Different Kind of Television). 

The service developed by Communicopia 
aimed to extend the reach of the NHS Direct 
telephone and online service onto digital 
television. It was launched (in November 
2001) to an audience in Kingston upon Hull 
and the East Riding of Yorkshire via Kingston 
Interactive’s (KIT) local ADSL telephone 
network, and later, on Video Network’s 
HomeChoice platform, to viewers in London. 
The service enabled TV viewers to access key 
sets of data from the NHS Direct Online Web 
site, including details of over 400 illnesses and 
medical conditions, support organisations, 
and advice about living a more healthy life. 
Utilising a mixture of existing text-based 
material and specially produced video clips, 
the service provided users with over 20,000 
pages of information, and video-on-demand 
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offering both the perspective of medical 
professionals and that of patients. The service 
also provided users with interactive options 
such as health quizzes and an SMS text 
messaging reminder service for children’s 
vaccination dates. This service was promoted 
as NHS Direct Digital. 

Flextech Living Health Ltd and partners 
launched a pilot service in June 2001 
featuring a range of digital TV health 
applications to a potential average audience of 
45,000[1] in the Birmingham area via the 
Telewest cable network. The applications 
included “NHS Direct in Vision” (talking to 
and actually seeing a nurse), a system for 
booking an appointment with a GP through 
the TV and a wide range (over 22,000 pages) 
of information pages covering local NHS 
services, healthy living, health conditions and 
treatments and a database of medicines. The 
pilot ran for six months and was then 
extended by Flextech Living Health for 
another six months. 

Channel Health piloted a series of 
broadcast TV programmes (Bush Babies) to a 
national audience of over 5 million with 
linked interactive services via Sky. The 
programmes dealt with health (and financial) 
issues relevant to pregnant women and the 
interactivity enhanced and supplemented 
these by providing a range of text-based 
pregnancy and maternity related information. 
Channel Health presented a text-based 
information service linked to special 
broadcasts in its regular schedule on the Sky 
Digital platform. It experimented, on a local 
basis, with a package of other interactive 
services for pregnant women comprising 
mainly e-mail support links between users 
and health professionals. 

DKTYV Ltd developed a service for the 
HomeChoice platform, launched initially to 
Newham residents. This service aimed to 
work together with public sector organisations 
to provide national and local interactive 
television content. The service relied on the 
HomeChoice video-on-demand, enabling 
users to select and find out more about the 
local services on offer. The video clips were 
presented by local service employees and 
provided users with the option to press a 
button to forward their details to local service 
providers. Originally intended to be rolled out 
on cable TV and broadband platforms in 
London, in the end only the broadband 
version of the service was launched and 
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evaluated. This service was transmitted to 
approximately 500 potential users on Video 
Networks Limited’s HomeChoice service in 
Newham. 


Aims 


These were: 

* To determine whether digital interactive 
television “worked” in a consumer health 
context. Did it, for instance, deliver the 
numbers, the particular audience profile, 
the ease of access and the hoped for 
health outcomes? 

* To evaluate and compare the four DoH 
contracted consortia’s approaches to the 
delivery of health information on DiTV. 
To establish which approaches worked, 
with whom, and which of the approaches 
worked best. 


The organisation of this paper follows the 

main performance indicators originally 

specified by the Department of Health in its 

briefing document. Most of these can be 

applied to each of the four consortia in turn. 

Indicators such as delivery of NHS services in 

a domestic setting by a commercial 

organisation, the costs of different DiTV 

services, and the effects of approaches used by 

broadcasters to combine NHS and non-NHS 

health programmes and material, are 

examined in a separate section of this report, 

taking all four DiTV consortia together. 

The key DoH evaluative criteria were: 

* volume and pattern of use; 

* users’ experience and perceptions of the 
service; 

* the range, quality and appropriateness of 
the services offered; 

* co-ordination with other health services 
available; 

* impact on NHS of supporting the service; 

* impact on users’ use of health services; 

* impact on users’ health status; 

* impact on their perceptions of the NHS. 


In addition, some comparisons were made 
between use of digital interactive television 
(DiTV), as a platform for health information, 
and other technologies such as touch-screen 
kiosks and the Internet. 

A number of factors made the meeting of 
the research objectives a difficult and 
challenging task because: 


139 


First steps towards providing the UX with health care information 


Aslib Proceedings 





David Nicholas, Paul Huntington, Peter Williams and Barrie Gunter 


The four consortia did not start at the 
same time, which made it difficult to 
compare the impact on each consortium 
of a health scare, seasonal illness or item 
of news. 

The periods over which the evaluations of 
the different pilots were conducted 
varied, giving some consortia longer 
exposure. 

Clearly, DiTV is a new platform that 
represents a development in the nature of 
television as a technology and as a 
viewing medium for the audience at 
home. The use of DiTV as a 
communications channel through which 
to transmit health information and advice 
— as evidenced in the current pilot 
exercise — is a pioneering innovation. 
Even on an established platform, it may 
take some time for a new service to 
become established in terms of its market 
share. In the case of the DiTV 
experiment, both the services and 
platforms were relatively new, not just in 
terms of content, but also in terms of the 
way users were invited to engage with the 
services being offered. While people may 
be accustomed to entering “interactive” 
mode when using the Internet, this is not 
true of television viewing. The long- 
conditioned psychological orientation 
towards television is one of passive 
reception — with interactivity limited to 
channel surfing via the remote control, 
video recording of off-air broadcasts for 
later viewing, and latterly, choice of 
camera angles at sports broadcasts. Full- 
on interactivity with the television that 
involves content mining or two-way 
communications, with the viewer being a 
sender as well as receiver of messages, is a 
new phenomenon Zor most viewers that 
will entail some re-conditioning of their 
orientation towards the medium. 

The four DiTV consortia offered 
different products to different audiences 
in different places — Birmingham, Hull, 
London, or, in one case, the whole of the 
UK. Some were mostly text-based, while 
others offered significant video material. 
Levels of interactivity varied from 
searching of text content — a kind of 
online encyclopaedia of health 
information, through video-on-demand — 
to personal medical records keeping, 
online appointments booking and two- 
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way, live and interactive, personal health 
condition diagnosis. All these distinctive 
features meant that we were not always 
comparing “like with like”. 

e Data quantity and quality varied between 
consortia. While it was originally 
intended to centre much of this 
evaluation exercise on computer log data 
that gave measures of real-time 
behavioural use of each service, this 
proved not always to be possible. No log 
data at all were produced by one 
consortium (Channel Health), while only 
summary data were made available for 
another consortium (DKTY). This 
meant that the comprehensive data 
comparisons we wished to make could 
not always be conducted. 

* Access to users or potential users also 
varied between consortia. One significant 
factor in this context (and also related to 
log data availability) is that the DoH- 
contracted consortia did not own or 
control subscriber databases for the 
various platforms on which their services 
were transmitted. The commercial 
organisations that owned the 
transmission platforms and controlled 
access to subscribers were not always 
willing to give access to commercially 
sensitive data from these potential digital 
TV health audiences. This meant that the 
data could not always be triangulated to 
best effect. 


In view of the above, the research team had to 
modify the original research design as certain 
methodological options and envisaged data 
sources were closed down. Where real-time 
log data were not available to measure user 
behaviour for these services, self-report data 
were obtained instead via ad hoc surveys with 
users or potential users. Where direct 
telephone access to viewers and users was not 
possible for lack of subscriber telephone 
number information or permission to speak 
directly to subscribers from platform owners 
and operators, postal or online self- 
completion questionnaires were used instead. 


Methodology 


A multi-method approach was used by design 
and by necessity. In the advance planning 
stages, it was recognised that the research 
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should attempt to study real-time user 
behaviour in relation to these DITV services, 
users reactions, and links between patterns of 
use and reactions and perceived impacts. 
Circumstances that arose during the course of 
the pilot exercise — and already outlined 
earlier — meant that standardised data could 
not be collected across all consortia. 
Nonetheless, the evaluative research 
maintained its commitment to assess use and 
user reactions as fully as possible throughout. 


Log analysis 

Analysis of digital logs was fundamental to 
this study. Logs were used to provide a 
bottom-line assessment of how much these 
DiTV health information services were uséd 
at all by people who subscribed to the 
television distribution systems that carried 
them. The great advantages of the digital logs 
are not simply their size and reach, although 
the dividend here is indeed a rich and 
unparalleled one. Just as important is the fact 
that they are a direct and immediately 
available record of what people have done: 
not what they say they might, or would, do; 
not what they were prompted to say; not what 
they thought they did. The data are unfiltered 
and speak for themselves. Log data enabled 
the measurement of overall numbers of users 
for the interactive services, trends in usage 
over the pilot period, the amount of time 
users spent visiting sites, and the types of 
content they viewed. Detailed log data were 
obtained in electronic form for two consortia 
— Living Health and Communicopia. In the 
latter case, such data were available for the 
use of text-based and video-based content. 
Summary log data only were provided for 
DKTYV and Channel Health generated no log 
data. 


Industry audience data 

Viewing statistics produced by BARB 
(Broadcasters’ Audience Research Board) 
were utilised in the case of one consortium — 
Channel Health. These data indicated how 
many viewers watched the consortium’s 
broadcast programme output linked to the 
project. A series of programmes targeted at 
women on maternity issues was broadcast as 
part of the regular Channel Health schedule 
on Sky Digital. Viewing of these programmes 
was measured by BARB’s audience 
measurement service. Viewing data are 
produced daily from a national panel of over 


Aslib Proceedings 
Volume 55 - Number 3 + 2003 - 138-154 





5,000 households. All TV sets and other 
related equipment, such as video-recorders 
and set-top box decoders, are linked to a data 
collection device installed in the home that 
automatically records which channel is tuned 
into when a TV set is switched on. Video 
recorded viewing is also registered. The 
presence of individual household members 
when a TV is switched on is recorded via a 
handset. 


User reactions 

Ad hoc user surveys (online, telephone and 
postal questionnaires) were carried out in 
respect of all consortia that were designed to 
supplement data obtained from real-time 
measures of viewing. These surveys 
questioned identified users of each DiTV 
health service about their experiences, usage 
and opinions. Questions probed issues such 
as ease of use and navigation, usefulness of 
information, the way information was used, 
perceived impact on own health or dealings 
with health professionals, and trust and 
authority of the information. 

With Living Health, data were obtained 
from two postal surveys conducted shortly 
after the service was launched and mid-way 
through the pilot period. Questionnaires were 
distributed along with promotional literature 
to Telewest cable TV subscribers in 
Birmingham and yielded return samples of 
450 and 723 respondents respectively. 

With Channel Health, data were obtained 
from two user surveys conducted respectively 
during the early part and the later part of the 
pilot period. The first survey was conducted 
by Channel Health in part by post (n = 114 
respondents) and in part via an online 
questionnaire (n = 322 respondents). The 
second survey was conducted in partnership 
with Channel Health by telephone interview 
(with fieldwork by Continental Research) and 
yielded 251 respondents. 

With Communicopia, a postal 
questionnaire survey was sent to all potential 
subscribers to the Kingston Interactive 
Television platform in Hull, and 1,184 usable 
responses were returned. 

With DKTYV, a single telephone interview 
survey was carried out shortly after the launch 
of its Newham service carried on the 
HomeChoice broadband network in London. 
At the time of the research, there were 513 
broadband subscribers in Newham and all 
were approached for interview. Interviews 
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were successfully conducted with 281, of 
whom 35 had reportedly used or watched the 
DKTV service. 


In-depth interviews 

The ad hoc surveys were supplemented in the 
case of most consortia with in-depth 
qualitative interviews with either users of 
DiTV services or health professionals 
involved with these services. 

With Living Health, depth interviews were 
conducted individually with six of the staff 
fronting the InVision service. In-depth 
interview data were also obtained from health 
professionals concerning the GP 
appointments booking service. Twenty 
in-depth interviews were conducted with 
Living Health users identified through one of 
the ad hoc surveys, four of whom had also 
used InVision. A further 31 InVision users 
were identified through caller logs, 27 of 
whom responded to telephone interviews and 
four participated in face-to-face interviews. 
Further qualitative data were obtained from 
14 Birmingham City Council employees 
about the Web version of the service carried 
on their employer’s Intranet. 

With Channel Health, qualitative data were 
obtained via the consortium among a small 
sample of pregnant women or young mothers 
and eight healthcare workers. These 
individuals were shown a demonstration of 
the Channel Health service and allowed to 
experiment with it themselves, before giving 
their impressions. Further in-depth telephone 
interviews were carried out with two pregnant 
women about the interactive services supplied 
by Channel Health. 

Communicopia held consumer panels once 
a month from June 2001 to explore issues of 
usability, information content and 
presentation. Each panel meeting involved 
between 10 and 15 participants, some of 
whom repeat attended. The sessions lasted 
between 90 minutes and 2 hours. Participants 
were presented with Cemonstrations of the 
service and opportunities to use it themselves. 


Usability research 

Serco Usability Services Ltd was 
commissioned directly by the Department of 
Health to conduct qualitative research with 
small, convenience samples who were given 
the opportunity to use service demonstrators 
in Serco’s observation laboratory. In all, 62 
respondents were run across the four 
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consortia. These data were examined 
alongside other data collected independently 
by the authors or in partnership with the pilot 
consortia. 


Results 


Use 

Over the 12-month pilot, the four DiTV 
pilots were made available to a potential 
combined audience of 5 to 6 million 
households, equating to 11.5 to 12 million 
individual viewers. The only nationwide 
service, Channel Health, attracted aggregated 
audiences of around 2.8 million to episodes of 
its Bush Babies series over the duration of its 
pilot. Based on self-report data, it is estimated . 
that 2-300,000 people availed themselves of 
one or other of the digital health services, a 
reach figure of 2 per cent. At first glance, this 
would appear to be a disappointingly low 
figure. However, why the reach figure is so 
small overall is because most of the base of. 
approaching 6m was contributed by Sky, and 
the reach of Channel Health in percentage 
terms was small (though it reached the largest 
potential audience overall). On Channel 
Health, 9 per cent of respondents who had 
identified themselves as viewers of Bush 
Babies reported using the maternity guide — 
the enhanced text service that supported the 
Bush Babies programmes. 

Hence the much healthier reach figures for 
Living Health and Communicopia of 30 per 
cent and 20 per cent respectively (see below) 
are significantly diluted by the performance of 
the service on Sky. Even DKTY, which 
operated on a smaller scale than any of the 
other consortia, achieved a reach figure of 35 
per cent. The specialised nature of some of 
the services (i.e. pregnancy information) and 
the short time-span of their availability also 
needs to be taken into account in any 
calculation. 


Individual service figures 

Living Health was made available to an 
average of 45,000 subscribers. Over the six- 
month period monitored, the logs tell us that 
13,718 different visitors (households) used 
the system. Based upon this figure it is 
estimated that around 30 per cent of potential 
users (i.e. “households”, as wherever 
mentioned in this report) accessed the service 
during the period — a healthy figure by any 
standard, which shows the level of penetration 
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these innovative health services are capable of 
achieving. This is especially true when one 
considers that, unlike daily news, for example, 
the kind of information/transaction offered 
was for ad hoc information needs. In 
consequence, subscribers may not have 
required any health-related services in the 
relatively short time of the pilot. The potential 
for growth appears there too, with well over 
eight in ten (84 per cent) of questionnaire 
respondents saying that they would be either 
likely to or very likely to access Living Health 
if the service continued to be broadcast. 
Interviewee respondents were equally 
enthusiastic: “It will definitely become a 
prime source of information for me if it carries 
on — and if it doesn’t I will be disappointed”. 
Although this finding did not quite square 
with the falling trend of use, the initial surge, 
and then decline, in usage may well have been 
due to its novelty value and early marketing 
efforts, and is probably unlikely to signify 
long-term stagnation. 

Communicopia (NHS Direct Digital) was 
available to approximately 10,000 potential 
homes. Over the five-month period 
monitored 1,965 different households used 
the system. Based upon this figure, it is 
estimated that about 20 per cent of potential 
users accessed the service during the pilot. 

In regard to Channel Health, BARB data 
indicated that Bush Babies episodes 
aggregated (across their repeat showings) 
audiences of over 300,000 in the first (six- 
week) phase, over 200,000 in the second 
(five-week) phase, and 2.3 million in the final 
(three-month) phase. According to self-report 
questionnaire data, 27 per cent of Channel 
Health viewers watched Bush Babies. This is 
quite impressive, given that the target 
audience was pregnant women, not only a low 
percentage of the population as a whole, but, 
presumably, a low number even of Channel 
Health viewers. Channel Health calculated 
that 160,000 out of 700,000 annual births in 
the UK could occur in Channel Health 
receiving homes. 

Channel Health is estimated to have a 
monthly reach figure among Sky viewers of 
approximately 15 per cent. Given a Sky 
Digital audience base of araund 5.7 million 
households then the audience of Bush Babies 
was estimated to be about 200,000 homes, or 
about 3.5-4 per cent of all Sky subscribers 
(assuming all Sky users receive 
Channel Health). 
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Finally, DKTV: during the two-month 
survey period 142 users availed themselves of 
the service, out of the 403 to 513 households 
receiving it — a reach figure of up to 35 per 
cent. 

In comparing the performance of the pilots, 
Living Health performed particularly well. In 
terms of audience reach, Living Health 
appeared to perform much better than the 
Communicopia/NHS Direct Digital, the 
service with which it had most in common — 
reaching an average of 30 per cent of the 
potential audience as compared to its rival’s 
20 per cent. Adjusting the reach figure for the 
different operational period lengths, Living 
Health is estimated to have a higher figure by 
about one-and-a-half times compared to the 
NHS Direct service on Communicopia. The 
reach figure for both services declined after 
about the 35th day of operation. 


Return visits 

The data show that once someone has used a 
DiTV health service there is a good chance 
they will use it again. About 60 per cent of 
Living Health users said that they would use 
the service as needed. In fact, the logs showed 
that 41 per cent of Living Health users visited 
the service again during the pilot. Living 
Health users who first visited the channel in 
July/August, shortly after launch, on average 
visited the channel three times or about once 
a month. Considering, again, the ad hoc 
nature of the need and the fact that everyone 
knew this was a pilot, these are quite 
impressive statistics. 

A slightly smaller proportion (37 per cent) 
revisited NHS Direct Digital over the period 
January to May 2002. It is also worth noting 
here that rate of return visiting may have been 
partly affected by (negative) changes to the 
positioning of Communicopia/NHS Direct 
Digital in the broadband environment in 
which it was transmitted. No logs were 
available for Channel Health, so the data are 
probably less reliable. However, subscribers 
were asked how often they watched the 
service and 56 per cent said they watched at 
least once a week, 20 per cent watched at least 
monthly and 24 per cent watched it 
infrequently. Return visits could not be 
calculated for DKTV. 


Time online 

Short online consultations proved to be the 
norm. Living Health viewers spent on average 
six minutes on a visit (session), whereas the 
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figure for Communicopia‘NHS Direct Digital 
was over seven minutes. Users viewing more 
pages in a session recorded longer session 
times. NHS Direct Digital users spent 
approximately 50-100 per cent more time on a 
visit as compared to Living Health users. This 
difference is thought to be due to the viewing 
of videos by NHS Direct Digital/ 
Communicopia users — video viewing is plainly 
more time-intensive. No data were available 
here for either Channel Health or DKTV. 


Screens viewed 

For the services for which logs were available. 
there was evidence that once people accessed 
a service they showed a significant degree of 
interest in it. In the case of Living Health, 39 
per cent of users viewed more than 20 pages 
during a visit and these figures proved to be 
stable over time. In the case of NHS Direct 
Digital, 44 per cent of users viewed more than 
20 pages, demonstrating, perhaps, a 
significant interest in what they saw. 
However, this service is a menu-heavy service 
and has a number of menu screens associated 
with both the video and text service. 

There were high “failure” rates too. In the 
case of Living Health, 19 per cent of users 
viewed one to three pages and these users 
were unlikely to have penetrated past the 
menu screens and would not have viewed an 
information page, except for the daily news 
headlines. For Communicopia/NHS Direct 
Digital the equivalent figure was also 19 per 
cent. Note that this service was very menu 
rich and to reach a number of information 
pages users had to go beyond three menu 
pages. 

Finally, logs also revealed that only 12 per 
cent of those accessing the Living Health 
InVision (Broadband nurse) service page 
actually completed a session (i.e. engaged in 
an online consultation with a nurse). These 
figures may provide evidence that early users 
were “checking out” the system for future 
reference, not having an immediate need. 


Personal characteristics of the DiTV 
health consumer 

Age/Gender 

In the case of Communicopia/NHS Direct 
Digital, younger respondents and, 
interestingly, especially male respondents 
were more likely to use the text service as 
opposed to the video service compared to 
older and female respondents. Men aged 55 
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and under were just over one-and-a-half times 
more likely to use the text service than women 
55 years and under. The relationship was also 
true of the video service. Furthermore, men 
over the age of 55 were about twice as likely to 
use the service compared to women under the 
age of 55. As for service preference, men 
reported a preference for videos over text. 

For Living Health, older users tended to be 
men. A higher proportion of younger Living 
Health users (compared to NHS Direct 
Digital) were women. In the case of the 
InVision home nursing service men and 
women used the service equally, although for 
different reasons. Women tended to do so on 
behalf of others (particularly children) whilst 
men wanted answers to their own health 
problems (maybe the privacy of the home was 
an incentive). 

Not surprisingly, given the nature of the 
Channel Health service (i.e. pregnancy 
information) those aged over 45 were four 
times less likely to use the service than 
younger viewers were. Women were twice as 
likely to use the service as were men. This still 
represents an encouraging interest in the 
service on the part of males. Interviewees all 
confirmed the interest of their partners in the 
programme and, indeed, how the content and 
information helped them understand and 
manage their situation: “we watch the 
programme together whenever we can — 
where (partner’s name) is at work, he watches 
on his own later — or I watch it a second time, 
with him”. 

Social class 

It would appear that DiTV attracts low- 
income users. This is encouraging in that it 
supports the argument that DiTV throws an 
ICT health lifeline to those who have been 
excluded from the digital revolution — the less 
well off and potentially socially excluded. 

People living in an area with a low incidence 
of £20,000+ income earners were more likely 
to use the Living Health service. Respondents 
from wealthier areas were half as likely to use 
a DiTV text information service, as were 
those people who came from less well off 
areas. In addition, users from lower income 
areas were more likely to say Living Health 
was useful compared to users from higher 
income areas. It was the same for the other 
services too. 

Communicopia/NHS Direct Digital users 
from postcode areas with a low incidence of 
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£20,000+ earners were about twice as likely 
to use the service. Channel Health users from 
middle and lower social classes were two to 
three times more likely to have viewed the 
service as were respondents identified as 
being from a higher social class. 


Children 

There was a high level of use of Living Health 
in areas where there was a higher incidence of 
0-14 year olds. This finding could not be 
confirmed among users of NHS Direct 
Digital on KIT or for those using Channel 
Health’s Bush Babies service. However, the 
researchers have found elsewhere (Nicholas 
et al., 2001), high levels of usage of other 
health information services (especially touch- 
screen kiosks) among the young (under 15s). 


Health concerns 

What health topics consumers decided to 
view or what medical conditions they wanted 
to look up is plainly of major interest to health 
information providers everywhere. However, 
we need to be careful how we interpret these 
data as choice is determined by a number of 
factors — what content is provided, how visible 
or accessible that content is, what health 
topics interest users and what medical 
conditions they — or their family — may suffer 
from. Comparisons between channels are 
difficult because of differences in the content, 
media, audiences and interactive services 
provided. The general indications were that 
users turned to DiTV health services to 
address real information/health needs rather 
than to browse for recreation. 

In the case of Living Health the sections 
that users appeared most interested in were 
the “Illness and treatment” section followed 
by “Women’s health” and “Men’s health”. 
The illness and treatment section accounted 
for 36 per cent of all pages viewed. The most 
popular topics in the illness and treatment 
section were back pain, depression, 
impotence, Aids, and irritable bowel 
syndrome. Popular topics under Women’s 
health were orgasm problems, dyspareunia, 
thrush and cystitis. For Men’s health 
interesting topics proved to be impotence, 
premature ejaculation, sexual infections, gay 
sex and sexual health help. Clearly, the 
privacy of one’s home appears to be the ideal 
environment in which to explore issues that 
may be too intimate or embarrassing to 
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research via a publicly placed information 
terminal. 

For Communicopia/NHS Direct Digital 
the “A-Z of conditions” was the most popular 
section by some margin and accounted for 57 
per cent of text pages viewed. The second 
most popular section was “Not feeling well”, 
and accounted for 13 per cent of pages 
viewed. Interviewees and questionnaire 
respondents for this project tended to stress 
their desire to avoid visiting a doctor: “If I 
find out that the problem is not serious I can 
avoid going to my GP. You have to make an 
appointment, it’s a long way, and the doctors 
are over-stretched anyway.” An established 
pattern of popular topics viewed emerged in 
the “A-Z of conditions” section from about 
February with diabetes, lower back pain, 
asthma, mellitus appearing in the top ten of 
subjects viewed in each of these months. 

For Communicopia, the most popular on- 
demand videos were the “Foray for health”, 
“Diabetes” and “Coronary heart disease”. 
These three videos accounted for 42 per cent 
of topic videos viewed. Each topic was 
represented by a series of videos that the user 
could view independently. For topics where 
only one video was available, the hypertension 
(downloaded 54 times) and MMR videos 
(downloaded 44 times) were the most 
watched. There were no views of videos on 
testicular cancer or ulcerative colitis. The 
latter might be considered disappointing to 
the DoH, as there is some concern that men 
are not undertaking regular self checks to spot 
this condition early. This raises the issue of 
whether topic positioning in the digital service 
should be used strategically to help ensure 
important topics are viewed. 


Role and impact of digital health 
information f 

Well informed consumers 

It would seem that DiTV creates well- 
informed health consumers. Two-thirds of 
Living Health users (67 per cent). said that the 
information they obtained had either helped 
or helped them a lot in becoming better 
informed about their condition. Data from in- 
depth interviews confirmed this, with 
respondents enthusing about having 
information so easily to hand: “It has given 
me a whole new source of information, and 
right there at home.” Similar findings came 
from Communicopia/NHS Direct Digital 
users; 90 per cent said they felt better 
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informed about a condition after having used 
the service. Pre-launch focus group 
interviewees felt that people would be, as one 
put it: “much better informed about health 
matters”. 


Helps with the doctor 

DiTV helps in dealing with the doctor. Well 
over half (55 per cent) of Living Health users 
queried the service for information about their 
consultation with the doctor either before, 
after or before and after their consultation; 40 
per cent of Living Health users felt that the 
information they found had helped or helped 
a lot in their dealings with the doctor. This 
was also a theme among depth interviewees: 
“T now go to the doctors with a much better 
idea of what’s wrong with me, and what I can 
ask about.” 

These are views echoed by NHS Direct 
Digital/Communicopia subscribers; 73 per 
cent of respondents felt that the information 
they found had helped a little or helped a lot 
in their dealings with the doctor. Just over one 
in four NHS Direct Digital users, 27 per cent, 
said that they would use the service to look for 
information that they would not want to 
discuss with their doctor. Plainly, this is an 
important role for DiTV health services. 

And the same was true about Channel 
Health users. Interviewees felt that they were 
far better armed when they met their GPs, 
midwives and obstetricians, and praised the 
Bush Babies series for giving them an excellent 
knowledge base regarding their condition: “It 
gave me so much information that I know I 
wouldn’t have got elsewhere — I don’t go 
fishing for information in books — and it’s easy 
to forget what the doctor tells you.” 


Used to put off a visit to the doctor 

Over half (53 per cent) of Living Health users 
confirmed that they had used information 
they found to replace a visit to the doctor. 
Those reportedly using the Living Health 
service were just under twice as likely to use 
information found as an alternative to seeing 
the doctor. Older users and men, however, 
were less likely to use information in this way. 
NHS direct online, health books and 
magazines and the Internet were also 
significant as sources used as an alternative to 
seeing the doctor. Importantly, the younger 
the respondent the more likely they were to 
substitute information for a visit to the doctor. 
Older users, those aged over 55, were half as 
likely to use information found on Living 
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Health as an alternative to a doctor’s visit, as 
compared to younger people. 

Where a person had an interest in general 
health, prescription drugs, healthy living and 
an interest in alternative health, then these 
people were more likely to use an information 
source as an alternative to seeing the doctor. 
This was strongly apparent in interview data 
as well as questionnaire returns (“I maybe 
know more about my health than the average 
person, and have a good idea about when I 
need to go to the doctors. I always do some 
research first now I have an extra resource.”), 
Indeed, Living Health was considered by 
subscribers as an important, if not a more 
important, source of information compared to 
the NHS Phone line. This has to be regarded 
as significant in the decision to roll-out digital 
health to the nation. 

Information on NHS Direct Digital also 
impacted on whether viewers used 
information found as a substitute for a visit to 
the doctor. Those using the service were just 
under one-and-a-half-times as likely to use 
information found in this way. This result was 
only significant at the 10 per cent level and 
further research is required to confirm the 
result. It was found, however, to be the least 
important information source compared to 
medical magazines and books, NHS Direct 
Telephone line and the Web. This suggested 
that the impact of this service was not as 
important as these other sources, probably 
because of its relatively limited content base. 


Leads to an improvement in health 
One-third of Living Health users said that the 
information found either helped or helped a 
lot in improving their condition. Nearly 62 
per cent of Communicopia/NHS Direct 
Digital said that the information found either 
helped a little or helped a lot in improving 
their condition and 15 per cent said that it 
had helped a lot. In the case of Channel 
Health, three out of four women who used the 
enhanced text service, supporting the 
broadcast programmes, said that they found 
its information reassuring. Interviewees 
tended to say that the way they approached 
their pregnancy was healthier because of the 
programme contents: “My diet was better, I 
exercised, I felt better about what was 
happening. The programme helped me in all 
of these things.” 

Whether the service improved the user’s 
condition or helped in understanding the 
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condition appeared to depend on the extent 
to which people understood the nature of the 
Living Health service. This points to the 
importance of design, navigation and 
documentation; possibly, also, digital literacy 
training. 

There was one surprise result. Whether the 
user found the system easy to view impacted on 
whether the person expressed a positive health 
outcome as a result of receiving digital health 
information, but the relationship was the 
inverse cf what we might have expected. Those 
finding the system easy to view were not those 
who felt that there were significant or positive 
outcomes resulting from use. There is a sense 
here that the harder you have to work at it, the 
more appreciative you are of the benefits. 


Use of the different digital products on 
offer 

The relative popularity of each type of service 
(text and video based services and interactive 
and transactional) was assessed. Health 
commentators have championed the video for 
consumer health information (especially for 
those with literacy and language difficulties), 
and their take-up was of particular interest. 
Two pilots in particular presented a 
significant amount of content in video format. 
Channel Health produced a broadcast series 
of TV programmes on maternity issues and 
Communicopia provided a video-on-demand 
service comprising material on a variety of 
health topics. 


Video Vs text 

Of Channel Health’s Bush Babies 
respondents, 70 per cent just viewed the video 
programme, 23 per cent reportedly viewed 
text and a video, while 7 per cent reportedly 
just viewed the text. 

Both forms proved popular to 
Communicopia/NHS Direct Digital users. A 
total of 41 per cent of those who had used the 
NHS Direct Digital service claimed that they 
only used the text service and that they had not 
requested a video; 6 per cent said that they had 
only used the video service and 54 per cent 
said that they had used both text and video. 

However, text proved to be the most 
popular; 60 per cent of NHS Direct Digital 
users said that they preferred the text service 
while 38 per cent said that they preferred the 
video service. This was reversed for men aged 
under 36. Focus group participants felt that 
some information presented in video form 
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would have been better as text: it was 
suggested that information on medication, for 
instance, could have been better presented in 
tabular form. 

Young men preferred videos; 63 per cent or 
men aged under 36 said that they preferred to 
watch videos rather than read text compared 
to 26 per cent of females in this age group 
who said so. 

Users of NHS Direct were also asked to rate 
the importance of each service, the health 
video and text service. The average scores 
were similar, though there is suggestive/weak 
evidence that videos are easier to understand 
(3.4) compared to text (3.3) and that videos 
are more interesting (3.3) than the text service 
(3.1). Focus group interviewees tended to be 
of this opinion: “Of course, it’s not just the 
information — if it is boring or dull, people 
won’t take it in. Watching a video is at least a 
bit more interesting.” Only 34 per cent of 
NHS Direct digital users, however, agreed 
with the statement that watching a health 
video was a big improvement on reading the 
text, one third disagreed and one third of 
users had no opinion. Men aged between 36 
to 55, and men and women aged over 55 were 
more likely to say that watching a health video 
was an improvement. Of this group, 42 per 
cent agreed compared to 31 per cent of 
females in the 36 to 55 age group, only 20 per 
cent of those aged under 36 (male or female) 
agreed with this statement. 


Transactional services 
These services explored the potential of DiTV 
as a two-way medium where the user becomes 
an information sender as well as receiver. 
Such applications represent more advanced 
forms of interactivity and require a different 
mindset on the part of users who engage in a 
customised activity geared to addressing their 
specific problems rather than ones of a more 
general nature. These are genuinely new 
applications of TV. The applications tested in 
the pilots included visual interpersonal 
communication with an NHS nurse 
(inVision), online appointments booking with 
one’s GP, and the maintenance of personal 
medical details online, in this instance 
personal immunisation records. In addition, 
one consortium (Channel Health) tested a 
small-scale e-mail support service for a 
specific group — pregnant women. 
(1) Living Health — InVision. Despite the 
obvious warmth with which both 
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consumers and nurses received the 
service, relatively few people chose to use 
it. A total of 163 users from a potential 
audience of around 38,000 subscribing 
households in four months appears low. 
Four possible explanations suggest 
themselves: a small potential user 
population; the discouragement of casual 
users by the channel itself; the lack of 
publicity for these services; and, possibly, 
most importantly, the novelty and 
unfamiliarity of the services. 
Nonetheless, the number of people 
(1,380) who activated pages leading to 
the “point of no return” connection 
button indicates much potential interest. 
Clearly, the issue is how you convert 
these “lookers” into users. 
(2) Living Health — GP surgery bookings service. 
Use of this facility was plainly very low 
with just 30 people making an online 
appointment with their doctor over a 
period of six months. This was partly to 
do with the fact that there were only three 
surgeries in the pilot, and that the 
surgeries did not “sell” the service 
sufficiently to their patients. Given the 
amount of work that would have been 
involved to fully sign-up to the project 
and that it would only be available for a 
period of six months, this was probably 
unrealistic. An open-ended (time wise) 
roll-out might have produced different 
results. 
Communicopia/NHS Direct Digital. Views 
to the vaccination service accounted for 
0.14 per cent of use; approximately 28 
people used it. The service consisted of a 
reminder of when a jab was needed and 
users had to enter all relevant personal 
details. The take-up can be regarded as 
disappointing, but one has to question the 
value of this to a consumer when they 
know that it will only be in place for a 
short period. 


(3) 


Usability 

Data were obtained from user reaction 
surveys, follow-up qualitative interviews and 
Serco usability lab research. DITV is thought 
to be an easy-to-use health information 
medium. In practice, however, usability of the 
pilot services was found to vary with the 
individual user and the nature of the service 
being used. 
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Of Living Health respondents, 28 per cent 
said that it was “very easy” to find the 
information. Just under half found it either 
OK, hard or very hard to find information on 
the channel. Interviewees complained of the 
number of screens required to navigate 
(“there is hardly any information on a page — 
even the shortest of topics has six or seven 
pages to it”). 

Most participants in Serco’s DKTV 
usability sessions were unclear about where to 
find the DKTV service within the 
HomeChoice menu on a broadband platform. 

Some services proved almost trouble-free. 
The interactive service linked to the Bush 
Babies television series on Channel Health 
was universally described as being easy to use 
(although even here there was some concern 
about using arrow keys and sub-menus). 
Interviews with users of the e-mail service 
indicated that they experienced difficulties: 
“Tt wasn’t Channel Health — it was the e-mail 
service that didn’t work very well.” 

Digital visibility the key 

In an analysis reviewing the menu position of 
the NHS Direct Digital on KIT (Nicholas 

et al., 2002), it was found that the service 
became more difficult to access as its sign 
posting became ever more removed from the 
television service’s opening menu. This led to 
the proportion of new visitors (as a 
percentage) of all users declining alarmingly. 
New users did not come through because of 
the increasing difficulty of finding the service. 
Those people who battled through to find the 
service, however, showed their tenacity by 
making more extensive use of the channel 
when they arrived. 

In laboratory-based usability tests, Serco 
found instances of poor positioning leading to 
a lack of use ~ DKTV’s “further details” 
button, for example, was missed by several 
viewers engrossed in the video content of the 
channel. 

In sum, the number of clicks to obtain 
content is a critical feature with DiTV 
interactive services, and determines whether 
people find what they need. 


Gender and employment 

Women were less likely to find the Living 
Health system useful “all of the time” 
compared to men. Women were also more 
likely to say that the information offered was 
full of medical jargon. 


148 


First steps towards providing the UK with health care information 
David Nichclas, Paul Huntington, Peter Williams and Barrie Gunter 





Users living in higher unemployment areas 
were more likely to say the Living Health 
system was easy to navigate. Clearly, this 
finding requires further investigation before a 
full explanation can be offered. 


Age differences 

The elderly are clearly a source of concern 
when it comes to DiTV health services. On 
the one hand they offer the prospect of 
convenience and doing everything in the 
comfort of one’s own home, but on the other 
they require a certain amount of dexterity and 
the elderly tend not to bring with them digital 
familiarity. 

Not surprisingly then, the age of the 
respondent was found to impact on how easy 
the user found it to understand the Living 
Health service. Users over 66 were more likely 
to report that they ran into difficulties. 
Furthermore, older NHS Direct Digital users, 
particularly women aged over 55, found the 
service difficult to use. 

Focus group sessions in which 
Communicopia/NHS Direct Digital videos 
were shown to elderly people revealed some 
anxiety about the march of information 
technology. Older participants felt that they 
would not have access to digital television in 
their lifetime and would be unsure of how to 
use it even if they did: “I can’t even use a 
video (recorder) these days — it’s got so 
complicated”. 


It helps if you are technologically literate or if you 
use a variety of health information sources 
Unsurprisingly, users with a greater 
experience of technology were more likely to 
find the navigation and menu structure of 
Living Health easier. Users reporting that the 
general KIT DiTV service was easy to use 
were more likely to have used the NHS Direct 
Digital service provided by Communicopia. 

Many interviewees who had used Living 
Health, or used the Bush Babies interactive 
service, or viewed the videos on NHS Direct 
Digital tended to use these in concert with a 
variety of other sources — Internet, books, etc. 
These information hungry respondents were 
used to various menu systems, indexes and 
different information configurations, and 
found it easy to adapt their skills to the 
medium of DiTV. (“I have always had an 
interest in health — now I can use the Internet 
and my television to supplement the books I 
have always used.”) 
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Contrasts between views of medical professionals 
and lay consumers 

The DiTV services were meant to be 
consumer friendly, obviously. Nevertheless, 
there is probably a tendency amongst health 
professionals to underestimate the consumer. 
Thus nurses who watched the NHS Direct 
Digital videos felt that the language and 
terminology used was often too difficult for 
the lay viewer. However, information 
professionals, general consumer interest 
groups and others all felt there was no 
problem in this area. 


Authority, trust and branding 

DiTV is a very new platform for health 
information and issues over whether the 
information could be trusted, and whether the 
NHS brand was visible and what it meant 
were of concern to the DoH. Clearly, matters 
are complicated by the fact that, with so many 
parties involved with the content, production 
and distribution of the digital services, 
ownership and responsibility are far from 
clear. 

Of the users on Telewest’s Birmingham 
cable platform, 60 said that they would trust 
the health information found on DiTV for 
most things; 38 per cent said they trusted the 
information and only 3 per cent of users said 
that they did not trust the information found. 
A substantial proportion (43 per cent) said 
they would not use the service should the 
NHS not be involved, and a large majority 
(81 per cent) thought the NHS should be 
involved with digital television, a finding 
echoed in interviews: “I just assumed the 
NHS was involved — it’s a public service isn’t 
it?” 

For certain types of people the NHS brand 
really meant something. DiTV subscribers 
who had either used the Living Health 
service, which carried NHS branded health 
information, or had heard of the service were 
more likely to say that the NHS was a symbol 
of trust than DiTV subscribers who had not 
used the service. There were, however, digital 
users who did not buy into the NHS brand. 
DiTV users visiting the doctor less frequently 
and those less interested in health information 
were less likely to accept the NHS as a symbol 
of trust, were less likely to recognise the NHS 
symbol, and were less likely to say that the 
NHS branded information could be trusted. 
Younger respondents were also less likely to 
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recognise the NHS as a symbol of trust 
compared to older respondents. 

Communicopia/NHS Direct Digital users 
were asked to rate the trustworthiness of 
health information sources out of 5. Sources 
where the NHS appeared in the name 
performed well — NHS Direct phone and the 
NHS Direct service available on KIT scored 
respectively 3.9 and 3.8. This score placed 
these services below doctors and nurses (4.5 
and 4.2) but above medical magazines and 
books (3.0). 

Around half of Channel Health viewers who 
were surveyed for their reactions to its service 
were aware of NHS involvement and their 
reactions to this were largely positive. NHS 
branding was seen as offering more authority 
and credibility to the content (“It was nice to 
know that this was real information from a 
proper source. It was obvious the programme 
was helped by the NHS”.) The more 
respondents had viewed the Bush Babies TV 
series, the greater their awareness of NHS 
involvement became. 


Satisfaction 

Satisfaction was a key DoH performance 
measure. They were not to be disappointed. 
A total of 45 per cent of Living Health 
viewers said that the service had been very 
useful, 48 per cent said that the service had 
been quite useful, while 7 per cent said that 
the service had not been useful. The main 
reason people gave for finding the service 
helpful was that they felt reassured by the 
information given — 50 per cent said so; 15 
per cent of users said that the advice helped 
them deal with the condition themselves 
while 20 per cent said that the advice helped 
them to contact the right service. Interviewee 
respondents also indicated that the service 
was very useful, enabling them to, for 
example, avoid visiting the doctor, research a 
condition for a friend, and check information 
about medication. If DiTV was rolled out 
nationally there would appear to be 
significant gains. 

The InVision service went down extremely 
well and comparisons between the telephone 
equivalent are instructive. Respondents rated 
the service either very satisfactory (76 per 
cent) or satisfactory (24 per cent) — 100 per 
cent satisfied in other words, versus 97.8 per 
cent (combined totals) for the NHS Direct 
telephone-only service. Additionally, 100 per 
cent of InVision customers polled said they 
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would use the service again and/or 
recommend it to friends/family — versus 97.8 
per cent for telephony. 

Finally, 88 per cent of those who had used 
both InVision and the telephony service 
preferred InVision (the rest had no 
preference). These results suggest that the 
nurses were able to offer an equivalent service 
while having to engage the camera and 
operate the InVision system in addition to the 
normal CAS (clinical assessment system) 
software. Interviewee respondents all rated 
InVision above the telephone only NHS 
Direct — enthusing about the facility to see the 
nurse (rather than about how images could be 
sent to them): “It was so good to see the nurse 
— it was obvious she was interested. With the 
telephone service you cannot tell how much 
she is paying attention.” 

The 35 respondents who claimed to have 
viewed or used DKTY were asked to rate 
their overall satisfaction with DKTV along a 
four-point scale from very satisfied to not 
satisfied at all. Two respondents were very 
satisfied, 21 were quite satisfied, seven were 
not very satisfied and one was not at all 
satisfied. The remainder did not know (or 
care). The average satisfaction score indicated 
that these respondents were only “quite 
satisfied” with the service. When asked how 
often they thought that they or their family 
would use DKTV in the future, most of these 
respondents (20) thought they would use it 
about the same as now, with nine saying they 
would use DKTYV more often and six saying 
less often. 

Most people who used the Channel Health/ 
Bush Babies service thought the service quite 
useful — 60 per cent said so. Those in social 
class D/E were less likely to say that the 
information would be useful for them: 17 per 
cent of this group said the information would 
be useful for them, compared to 40 per cent 
from other groups. Furthermore, those living 
with a partner said that the videos were useful 
to them: 56 per cent said this compared to 37 
per cent of married and 8 per cent of single 
users. For those users who were pregnant, 83 
per cent said the information was useful to 
them compared to 29 per cent who were not 
pregnant. 

People who used Communicopia/NHS 
Direct Digital generally found what they were 
looking for. A total of 15 per cent of NHS 
Direct viewers agreed and 69 per cent 
disagreed with the statement that in general 
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users could not find what they are looking for 
on the service. Of the users, 81 per cent said 
that if they needed health advice they would 
consult health information on KIT. There is 
suggestive evidence that carers (informal and 
formal) were less likely to say that they would 
consult the service compared to non-carers. 
This needs further research, particularly as it 
is contrary to, what one would expect. 


Comparisons with other digital 

platforms 

In parallel with the DiTV study, City 

researchers were also investigating other 

digital platforms providing heath information 
and advice[2]. On the basis of this work 

(Nicholas et al., 2001; Williams ez al., 2001, 

Huntington et al., 2002), the following 

comparisons have been made: 

* Compared to the Web and kiosks, DiTV 
attracted a greater percentage of its users 
at weekends. This is no surprise, as the 
DiTV service is the only “home only” 
service. Many people may only have 
access to the Web at work or in a public 
place, and the kiosks are, of course, sited 
away from the home. 

* Compared to the Web, DiTV attracted 
more repeat visits. This could be ascribed 
to easier access, but it could also be a 
function of more limited choice — it is 
possible that Web users revisited for 
health purposes, but to another 
competing health site. During the pilot 
period, there was no competing health 
information service on any of the specific 
DiTV platforms being investigated. 
Hence, Telewest subscribers, for 
example, had no option but to return to 
the Living Health channel. 

«+ DiTV users took a third longer to view a 
screen than kiosk users, possibly 
reflecting the fact that they were using 
the service in the comfort of their own 
homes. 

*  DiT'V users viewed three times the 
number of pages that kiosk users did. 
This is because DiTV users have the time 
to penetrate the system. Furthermore, 
users cannot scroll on DiTV and will 
inevitably view more pages. It is not 
uncommon to find health information, 
but not menus, spread over up to seven to 
eight pages. 

* DiTV users spent the most time viewing — 
approximately five to six minutes during a 
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session compared to three to four minutes 
for Web users and about one minute for 
kiosk users. This is thought to be because 
DiTV users had more time to view pages 
and, in the main, were forced to read the 
content, as they did not have a printer 
(some KIT users had printers attached to 
their TV sets). 

* Compared to kiosk users, DiTV users 
were more willing to interrogate the 
system beyond the menu screens. Kiosk 
users were thought to be rushed for time 
and may have been unwilling to invest the 
time necessary to interrogate what may be 
a new system for them. As previously 
mentioned, DiTV users view the service 
in their home, at a time convenient to 
them, and on a system that they have 
used before and hence may be far more 
willing to interrogate the system. 


This all suggests that, because users are 
viewing a system in their own home and one 
with which they are familiar, they will 
interrogate more topics, view more pages, 
will have longer search sessions etc. The log 
metrics point to a greater use of DiTV. To 
date, the data are lacking to give a 
comprehensive analysis of the reasons why 
such differences in use of online health on 
different platforms occur. However, it seems 
highly likely that differential ease of use of 
the technologies and information systems is 
one factor. Clearly, with kiosks, users may 
not have seen the system before and they 
might be at a loss to use the service 
effectively. This is not so true of the DiTV 
service, where users have some previous 
relevant experience in the use of the 
medium. In fact, our research generally 
points to greater use in households where the 
TV plays a central role — i.e. where 
households have a previous experience of TV 
viewing. If you like, the technology learning 
curve is lower. In relation to the use of 
television interactively, however, industry 
research has indicated that significant 
numbers of subscribers to mainstream 
television services offering interactive 
capabilities do not use the interactive 
elements of these services. In any case, 
interactivity on mainstream broadcast 
television tends to occur at an elementary 
level (e.g. choosing between different camera 
angles at a sports contest or selecting 
enhanced news footage behind a regular 
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news broadcast). The more complex level of 
interactivity represented by the digital health 
pilots requires a different degree of 
involvement on the part of viewers. 


Overall conclusions 


DiTV appears to have the potential to attract 
a large audience. This audience comprises an 
extensive number of occasional users rather 
than a large one-off audience, but that 
probably reflects the nature of the health 
information need. Reach figures indicated 
that significant numbers of people who have 
access to these online services use them from 
time to time. Encouraging take-up figures 
suggest a viable future, and evidence shows 
that when people use the services once, they 
are likely to do so again. 

Health is a popular topic likely to attract 
viewers to watch regular television 
programmes, but it can also encourage 
viewers to use television more interactively. 
They are prepared to search for information 
that is of use to them or their families. It 
should be noted, however, that they might not 
be prepared, as yet, to search as much for 
health information as Internet users. Hence, 
interactive TV sites must be easy to use and 
enable users quickly and efficiently to reach 
the information they seek, and the content has 
to be worth seeking. 

What is also encouraging is that the digital 
health services appear to reach groups that the 
DoH have targeted because they have proved 
that can be difficult to access via other 
methods. Furthermore, DiTV has achieved a 
different demographic profile of users from 
that of other platforms, which shows that a 
suite of online technologies is needed rather 
than all the investment going to a single, 
supposedly convergent technology. 

As well as low-income groups, DiTV 
reaches older users, especially older male 
users. There is also evidence that it can 
encourage younger male users who are noted 
for not checking up on their health. It can also 
be effective at reaching specific groups with 
specific conditions, e.g. pregnant women. 

Low take-up of Living Health’s InVision 
and online GP bookings service suggests 
unfamiliarity and an uncertainty with 
transactional services. The level of use 
recorded for Communicopia/NHS Digital 
Direct’s immunisation records service 
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reinforces this point. This may indicate a need 
both for user friendly interface designs and 
possibly also training designed to promote 
interactive TV literacy. Alternatively, it may 
also indicate that some applications, such as 
an appointments booking service, where a 
degree of negotiation and flexibility are 
required among patients and health service 
providers, are more difficult to transfer into 
the online environment. 

People appear to be selective in their use of 
different media. Pages and topics accessed 
via DiTV are different from those accessed 
on the Internet or a publicly located touch- 
screen kiosk, due to considerations of 
privacy, time availability and the amount of 
choice offered. With regard to format, video 
appears to be less consulted than text 
services. However, it might be the case that, 
while the absolute number of videos 
consulted is less than the absolute number of 
text pages examined, users spend more time 
with videos than text. So overall, distribution 
of time to these formats might work out 
about the same in the end. 

Again, there is the issue of “application 
effectiveness”. This means finding the most 
appropriate, effective and acceptable format 
for a particular type of application. Focus 
group interviewees suggested that text was 
more appropriate for certain kinds of 
information (e.g. basic facts), and more easily 
consulted than having to run through a video 
to find a particular item. Video has a different 
psychological functionality for users. Text is 
factual, while videos can succour emotional 
needs too. This point is important both in 
respect of the construction of online libraries 
of health information, and also in relation to 
“live” transactional links. There is mounting 
research on the use of computer-mediated 
communications — outside the health sphere — 
that has shown that the rules of interpersonal 
communication online may vary from those 
that prevail in face to face communication. 
This research may have important 
implications for the design of user interfaces 
and decisions about the types of formats that 
could most effectively be applied to facilitate 
online health-related communications. 

Even where there was no facility (i.e. 
printer) to copy information, time spent 
perusing pages has to be regarded as 
minimal. Further work is needed to examine 
how information is assimilated and used by 
DiTV subscribers. For instance, users may 
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feel better informed by video, but absorb less 
factual information than they would from 
text. 

The health topics viewed, whilst differing 
across platforms and formats, nevertheless 
appear to suggest the services were used very 
much for consultation/reference with regard 
to specific conditions, rather than for general 
browsing or recreational use, which one might 
have expected. Users of Living Health 
acknowledged that this was a primary reason 
for using the service. Digital health services 
provide support for a number of specific 
functional needs on the part of NHS patients 
and members of the general public. Future 
research should attempt systematically to map 
out these needs and determine the delivery 
platforms, formats and content that will 
effectively satisfy them. 

The information retrieved appears, for all 
services, to have made some impact on 
dealings with doctors, and there is some 
evidence to suggest that people are using it as 
an alternative to making GP appointments. 
Our data provide support for the belief that 
digital health services could help the NHS in 
terms of reduced demand or economies in 
relation to certain offline services. 
Furthermore, and in a broader sense, if online 
health information can cultivate greater self- 
care and adoption of preventative 
“medicine”, further economies may be felt in 
an over-stretched NHS. 

Self-report data indicate that having 
information does help one manage medical 
problems, in a conditional way. The degree to 
which this occurs also depends on the type of 
user. For example, Living Health users stated 
that the type of service it provided could help 
them in relation to consultations with a 
doctor. There was further evidence that it 
could serve as an alternative to information 
from a doctor — though non-users indicated 
that they preferred getting medical 
information straight from the doctor, whether 
spoken or in writing, than any other 
information source. 

For the pregnant women of Channel 
Health, the TV/interactive service was seen 
mostly as a supplement to seeing health 
professionals. With their particular condition, 
it was still essential to have physical diagnoses 
and checks with doctors and nurses, etc. 
There was some indication that a service such 
as Channel Health’s interactive elements 
could be used in relation to reminders of 
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appointments — as endorsed by health 
professionals. However, as the Living Health 
experience demonstrated, there is a relatively 
slow take-up for an actual online 
appointments booking service. 

Summing up, the DiTV online health 
service pilot delivered mixed results across a 
variety of services. Early audience uptake of 
these services indicated that a market for 
digital health does exist. The public are likely 
to welcome an online health information 
service sourced by a trusted brand. Health- 
related programmes, especially on themes of 
special interest to niche audience groups, will 
attract viewers, as will health-related videos 
on demand, provided they are of good 
production quality. Enhanced information 
services that lie behind familiar broadcasting 
will also be accessed, provided the user 
interface is not overly complex. More 
sophisticated interactive services that invite 
two-way flows of information between the 
user at home and health service provider are 
likely to take longer to become established. 
In popular parlance, television is a “lean 
back” medium that has not traditionally 
invited its users to actively engage with it. In 
contrast, the personal computer is a “lean 
forward” medium that has always 
encouraged a high degree of interactivity on 
the part of its users. 

Different technologies are associated with 
distinct psychological dispositions on the 
part of their users. This distinction must not 
be forgotten when considering television’s 
potential as an interactive technology. Over 
time, these two technologies may become 
largely indistinguishable, as will the 
psychology of users in each case. Until then, 
any application of television as an 
interactive health information medium must 
be mindful of these distinctions and take 
them into full consideration when 
determining the nature and form of digital 
health services on the box. 


Notes 


1 The Telewest digital household figures for 
Birmingham were: 1st June 2001: 38,149; 
1 December 2001: 51,963. This provides an average 
household figure across the pilot of 45,056 homes. 

2 The Web, the kiosk, digital TV and the changing and 
evolving face of consumer health information 
prevision: a national impact study, 2000-2003. DoH 
funded, 
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This artide reviews a significant range of new products 
developed by library management system (LMS) suppliers, 
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other library stock. The article reviews current 
understanding of user requirements and then describes 
the state of the current market place, based on a survey 
of suppliers conducted in May and June 2002, as well as 
on supplier Web sites and live demonstrations. The article 





concludes by summarising evaluation criteria for choosing 
“a system, and makes some tentative predictions about the 


development of the market place in the next five years. 
The perspective is primarily that of a UK academic library, 
but the products are certainly of interest to libraries in 
many sectors in every country. 


Electronic access 


The Emerald Research Register for this journal is 
available at 
http://Awww.emeraldinsight.com/researchregister 





The current issue and full text archive of this journal is 
available at 
http://www.emeraldinsight.com/0001-253X.htm 





Aslib Proceedings 

Volume 55 - Number 3 - 2003 - pp. 155-165 
© MCB UP Limited - ISSN 0001-253X 

DOI 10.1108/00012530310472642 


The problem: integrated access and 
management 


To date, full text electronic information 
sources, e-journals but also electronic books 
and locally digitised content and links to free 
resources have tended to be managed in UK 
academic libraries separately from 
“traditional media” such as books, journals 
and other stock. This followed practice with 
bibliographic databases, which were 
catalogued only in the LMS at the database 
level. When online public access catalogues 
(OPACs) were primarily searched through 
text based interfaces the value of cataloguing 
individual electronic journal titles was limited, 
as the user could not satisfactorily click 
through to the full text. But this factor has 
declined as increasingly all access to the 
catalogue is through a webpac. Another 
obstacle to cataloguing electronic collections 
in the library catalogue was the changeability 
of URLs, particularly as library systems did 
not have global replace features to allow 
changes to base URLs to be made easily. In 
some publisher systems URLs for individual 
e-journal titles could not be specified. 
Gradually, though, publishers and 
aggregators have recognised the value of 
specifying persistent URLs, down to the 
article level, and of supplying libraries with 
ready-made MARC cataloguing data for 
individual e-journal titles. Cataloguing e- 
content was also seen as diminishing the 
quality of the library catalogue, which 
traditionally was of tangible objects the library 
owned, rather than of objects for which a right 
of access was licensed for a distinct period. 

So for a mixture of reasons users have been 
directed to search information collections of 
e-content outside the OPAC. But as academic 
libraries have bought more and more 
electronic information sources and larger and 
larger collections of e-journals, the desire to 
offer users the ability to find electronic titles, 
or check whether a title is available 
electronically as well as in print, has 
encouraged libraries to seek to manage access 
to e-content in a more integrated way with 
other library holdings. The complexities of 
searching multiple sources including the 
library catalogue, multiple e-journal 
collection interfaces, and multiple 
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bibliographic databases to find references and 
locate the full text, reflected in inefficient use 
of document delivery and ILL, has come to be 
seen as a critical problem for user education. 
It is seen to lie behind students’ neglect of 
quality resources and the preference for 
simple google searches, however inadequate 
this is in terms of quality (Armstrong et al., 
2001). In short, libraries are searching for 
tools to help users discover the appropriate 
resources to search, to conduct the search and 
to guide them to locating the full text, be it 
online or in a library. Equally, the 
management facilities, such as budgeting and 
reporting in the LMS for books and journals, 
could usefully be applied to electronic 
content, ideally in an integrated way. 


Library controlled solutions 

A number of solutions have been developed, 
on a rather ad hoc basis, to manage e-content 
for users. They have focussed primarily on 
e-journals. The simplest approach has been 
that many libraries have created static Web 
pages, not simply listing major collections that 
are available to users, but all the individual 
electronic titles (often running to hundreds) 
they have access to. This listing might show 
years available, a link to help or authenticate 
information. The problems with this 
approach are first that the user has to search 
the listing separately from the OPAC and 
other sources. Nor does it, second, offer the 
user the access points they want, e.g. searches 
by subject or at the article level. Third, in 
management terms major changes to the list 
are very time consuming to make. A practical 
solution to the last problem has been sought 
in creating locally designed Web enabled 
databases of e-journal titles and other 
electronic content. This makes it practical to 
quickly update journal level links and show 
subject clusters, not just list e-journals in title 
order. To a certain extent it is a future 
proofed solution in that descriptive data held 
in the database could be exported to another 
system (Gardener and Pinfield, 2001). The 
two main limitations with this approach are 
that first there is no linking of titles in the 
OPAC and electronic versions so that users 
continue to have two places to search for 
relevant sources. Second, and more 
fundamentally, searching is at the journal title 
level not at the article level. A solution to this 
problem is offered by some third party 
services such as TDNet and Serials Solutions 
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(for URL see references). TDNet is a third 
party service for managing a library’s 
e-journals and offering TOC and article level 
searching. The library supplies TDNet with a 
listing of e-journal titles held and their print 
titles, and from this TDNet constructs a 
searchable database of all e-titles customised 
for the individual library, with links to 
holdings statements in the OPAC where print 
versions are held. Because TDNet is 
managing the same titles for many libraries it 
is practical for them to add article level detail; 
and based on this TOC alerts. TDNet 
supplies a list of electronic holdings suitable 
for importing into the local library system. 
There are perhaps small weaknesses, for 
example, in the level of statistics; but the 
primary remaining problem with this 
approach is that it requires users to search 
separately for e-journals outside the library 
catalogue. 


New solutions from LMS vendors 


During 2001 and 2002 a range of new 
products have been launched by LMS 
suppliers. Three main areas of functionality 
can be identified in these systems, with 
different systems stressing one aspect more 
than another (although the underlying 
technology may be similar): 

(1) Managing the digitisation process, 
storage of digital content and managing 
access to digitised material, including 
images. 

Bibliographic enrichment, that is adding 
links to or incorporating book tables of 
content, reviews, dust cover images and 
other enhanced information into 
traditional descriptive records about 
library holdings. 

Managing and integrating information 
from multiple suppliers with multiple 
hosts/metadata formats in use. 
Supporting resource discovery and 
searching, and document delivery. 


(2 


Ve 


(3 


ww 


It is this third function that meets the need to 
manage e-content alongside books and 
journals, and it is with this function the main 
body of this article is concerned. 


Requirements 
An awareness of the requirements of an 
academic library from such systems has been 
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developed over the last five years, most 
notably in eLib’s Hybrid library projects and 
EC Telematics for Libraries projects (see 
especially HeadLine, 2001; Russell, 1999). 
Indeed two of the systems trace their roots 
directly to that work: OCLC/Pica’s iPort is 
based on the system designed in the 
Decomate project, and ZPortal is based partly 
on software designed in the Agora project 
(Russell, 1999). Interestingly, systems 
originating in the USA to date appear to be 
almost always created by a single vendor 
working with their customers rather than by 
wider collaborative consortia. 

The requirements could be summarised as 
follows: 


Tools for resource discovery 

Tools are needed for the user to navigate 
through available resources, e.g. by searching 
descriptions of services or browsing resources 
pre-clustered by subject. 


Cross searching 

The ability to cross search different resources 
is required, that is first across metadata held 
in different repositories, second, across 
content described by different metadata 
standards (including across curatorial 
domains), third, using different search 
protocols and fourth, finding different data 
typés (e.g. images and full text as well as 
bibliographic information). Results from such 
searches could create conceptual problems in 
understanding, for the user, if not handled 
carefully. Cross searching just anything does 
not make much sense, and puts greater stress 
on the users’ ability to evaluate the 
information thus found, than searches within 
a single well known source. It might, however, 
be satisfactory for simple searches, by “naive 
users”. After all, this sort of bucket search is 
essentially what search engines do. More 
meaningful results will be found if datasets are 
pre-clustered by subject. 

The systems now on sale rely primarily on 
Z39.50 to search multiple data sources. There 
are known problems with exchange of data 
through Z, such as poor maintenance of Z 
servers, variable quality of basic data except in 
the most core data fields. Cross searching will 
not work well if the data in the target system 
are poor or inconsistent. Serial holdings are 
catalogued differently in different suppliers’ 
systems. Searches are often slow. Some of 
these problems will begin to vanish as cross 
searching becomes more commonplace and 
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errors and weaknesses in data are exposed and 
recognised to be important. More 
fundamentally, Z39.50 may itself have a 
limited future since it is often perceived to be 
over complex and has not been universally 
adopted. A promising alternative is Web 
services technology (see Gardner, 2001; 
Powell and Lyon, 2002). The typical model 
used in LPS is of distributed searching, that 
is, a search query is sent on the fly to various 
data sources, and the results returned for 
processing into a results report for the user by 
the home system. OAI partly challenges this 
and proposes that it is more efficient to 
harvest metadata to local systems. Mixtures of 
these approaches are also feasible. 


A common interface 

Users need a common, familiar search screen 
that could be used to search any dataset, as 
well as the facility to switch to the native 
search interface where appropriate. While 
searching multiple databases from one screen 
has its problems, the benefit of having a 
common interface that is familiar to the user 
and can work fairly well with many systems is 
attractive. Again this is a feature offered by 
LPS. Users can be trained to use the one 
system, and then move comfortably between 
datasets. It remains true, however, that 
publisher interfaces have powerful search 
functionality, especially where they offer an 
advanced taxonomic structure with which 
users are familiar and have been trained to 
habitually use. It is unlikely that the common 
interface will be as good as the database’s 
native search interface in the near future, 
especially regarding the taxonomic features. 
So it is essential that the user still has easy 
access to this interface when required. 


Linking and support through to document 
delivery 

Cross-linking between resources is desirable 
as well as cross searching, for example, so that 
one can find the full text of an article 
discovered through online citations. 
Resolution should find the “appropriate 
copy”: that is, the best of all document 
delivery options (for example, go to library, 
full text online, possibly in multiple places, 
ILL, document delivery) for that particular 
user, as determined by rules set by the library. 


Authentication/authorisation 
Ideally the system should interact with other 
local authentication systems so that the user 
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only has to log in once somewhere on the 
local system for authorisation credentials for 
other systems to be activated. This will 
achieve a “single sign on”. (Pinfield et al., 
1999). Having to remember multiple 
passwords is a common complaint cited as a 
reason for not using library approved 
resources, as opposed to simple google 
searches. The full functionality of the system 
should be available off campus. 


Profiling of users 

Profiling of users enables users to save 
searches, rerun searches, register to be alerted 
of new TOCs and to customise interfaces or 
prioritised resources. 


Interoperation 

A fundamental need is that the system should 
be capable of interoperating with other 
systems, based on open standards. Just as the 
library portal serves to integrate content for 
the user from multiple sources, so the library 
portal may itself be composed of functions 
provided by services from a number of 
systems: resolvers, authentication brokers, 
agents. The solution to creating a Web of 
interoperating systems (which appear as one 
to the user) is standards based 
“Middleware”. Rather than creating bespoke 
links between one system and another, it will 
be far more efficient to base all the systems on 
standards and so be able to design generic 
links (Paschoud, 2002). Services can be 
integrated by libraries themselves, but the 
nature of the technology invites arrangements 
between system suppliers. Thus there have 
been many deals between system vendors and 
suppliers of enhancements to bibliographic 
data. At its launch in mid 2002 Talis offered 
TDNet as part of its Information 
Environment. 

Just as the library portal pulls together 
multiple sources and services, it should also 
be able to offer a service to other systems. 
Although the library would like users to know 
they are using library supplied services when 
they do so, and to come through the library’s 
own portal to the world of information, in 
reality there are many competing “gateways to 
everything”. The library will commonly not 
be the most used starting point for a literature 
search. It is far more likely that a student wiil 
see “home” as an institutional portal or a 
virtual learning environment or a 
departmental Web site. This means that it wiil 
be most effective for libraries to offer up the 
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library portal to supply a service to other 
gateways. Thus a library managed collection 
of links to subject resources could be inserted 
within a departmental Web site, or searched 
within a VLE. Web services technology 
increasingly makes this possible. There are 
some problems with achieving this, especially 
where the library is first to develop its system, 
since it must be future proofed against 
options chosen by the institution later. The 
technology used may be an issue, but so may 
be subject terminology. Library resources 
must be accessible by subject, but the subject 
terms in use and the basic divisions must 
match institutional understanding of how 
study subjects are divided up. An institution 
may produce this after the library has settled 
on its own. 


Statistics and management information 
Statistics about users. and usage should be at 
the heart of improved management of 
electronic information. Currently, libraries 
have huge problems measuring the relative 
performance and value of services, because 
each publisher offers usage data in multiple 
formats there are no agreed definitions of 
terms for different fields. Often usage data are 
offered statically on the Web, without the 
ability to export it to spreadsheets for 
processing and cross analysis. A single library 
may well have 50 or 60 usage reports, each 
differently formatted. Part of the promise of 
library portal systems is that they will 
integrate and unify this management 
information. 


Look and feel 

The look and feel of a search interface is 
critical to its acceptability, especially with 
increasingly stringent usability and 
accessibility requirements being applied in the 
academic sector. The concept of skins should 
allow suppliers to deliver improved interfaces 
over time and information professionals to 
customise local implementations where 
necessary. XSL stylesheets and 
transformations are used to allow separate 
development of content, workflow and 
presentation that can meet a variety of end- 
user needs, including those with desktop 
terminals, portable devices such as PDAs and 
laptops. This avoids the need te re-key 
content, and allows libraries to give more 
responsibility to users to keep their own 
personal information up to date. 
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Communication/real time support 

ARL requirements include provision for real 
time support tools, recognising that scaleable 
support to distributed users is required. Also 
on the Web, portals are often used to help 
develop “virtual communities” or loose 
networks between people with similar 
interests. Academic institutions have often 
been poor at this sort of knowledge 
management, connecting people with similar 
interests who happen to work in different 
departments. The LPS could be the focus of 
creating such communities, be this through 
connecting people who have similar search 
profiles, collaborative surfing or even through 
controlled public exposure of reference 
enquires (Robertson and Reese, 1999). 


Core technology 

In theory, at least, if based on programming 
languages such as Java, such a system should 
work on any platform. This would be 
advantageous as most institutions have a 
preferred operating system or flavour of unix. 


Some characteristics of the market 
place in 2002 


During 2001 and 2002 almost all the larger 
LMS vendors brought out a product that 
sought to satisfy the requirements described 
above. The following observations of the 
character of the market place are based on a 
survey of the suppliers conducted in May and 
June 2002, as well as direct discussions with 
the suppliers and a reading of product 
literature and product Web sites and 
“canned” demos (Cox and Yeates, 2002). 
The authors acknowledge the limitations of 
this as an exercise, and offer the following 
discussion to stimulate open debate rather 
than to guide system choice, which must 
necessarily be a more rigorous process based 
on detailed evaluation of current systems and 
systematic trials of demonstration systems. 
Naturally, it is no intention of the authors to 
either endorse any product or do other than 
fairly represent each supplier’s market 
position. A list of the suppliers surveyed is to 
be found in Table I. 


Market leaders 


In mid 2002 Ex Libris and Fretwell-Downing ~ 


Informatics (FDi) could probably have been 
considered to be market leaders, by virtue of 
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recent major sales in the USA, Fretwell- 
Downing to some ARL member institutions, 
as part of the Scholars Portal project 
(Fretwell-Downing, 2002), and Ex Libris 
through sales of SFX (not Metalib as such) to 
the California State Library system (Ex 
Libris, 2002). Endeavor also has a highly 
developed product, ENCompass, and 
Innovative’s MAP has been in place for a 
number of years, a fact that is masked by it 
only recently being “productised”. These four 
suppliers have the largest claimed sales to 
date, though some of the sold systems may 
not yet have been implemented. Metalib is 
claimed to have been sold to 40 organisations 
worldwide, chiefly in the USA, with SFX by 
itself having been sold to 190 institutions, in 
17 countries. FDI claims 14 sales for Zportal, 
counting ARL libraries as individual 
organisations. Encompass has 24 sales, 
mostly in the United States and MAP 26. Just 
a handful of all these sales have been in the 
UK. The vast majority of ENCompass and 
Zportal customers are US academic libraries. 
Metalib’s customers are noticeably more 
diverse with a pharmaceutical company, 
national library and sales in Europe, Australia 
and South Africa, but also to Harvard and 
MIT. Other suppliers have much smaller 


‘customer bases to date, for example, PICA 


has sold to only five universities (all in 
continental Europe). The original 
requirements for Metalib were set out by 
KOBV, the Cooperative Library Network for 
Berlin and Brandenburg, as far back as 1997. 
Most of the other products seem to have 
much shorter life histories. At the same time, 
while customers may be reluctant to buy a 
raw product, today’s market leaders can be 
quickly overtaken in a rapidly changing 
technical environment, Talis and ESP’s X 
products speak much more boldly the 
language of J2EE and Web services, while 
Zportal, say, is closely tied to the language of 
Z39.50, as the name suggests. 


Interoperability 

Nearly all the products are marketed as being 
capable of being used as standalone from any 
particular LMS, and capable of interacting 
with any LMS. Metalib has been 
implemented with other LMS including 
Innovative and PICA systems, ENCompass 
with SIRSI and innovative sites, Zportal with 
a large number of other systems. In other 
cases the claims to work with other LMS may 
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Table | Supplier details 
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Product name (supplier) Type Web Address 
Chameleon iPortal (VTLS) Supplier: http:/Awww.vtls.com/index.shtm] 
Product: http://www.vtls.com/Products/gateway/ 
Demo: http://hermes.vtls.com:8000/cgi-bin/gw_40_2/chameleon 
DigitaLink/iPac (epixtech) Supplier: http:/Awww.epixtech.com 
Product: http://www. epixtech.com/products/ipac.asp 
Demo: http://Awww.epixtech.com/streaming/ipac/demo.asp 
ENCompass (Endeavor) Supplier: http:/Awww.endinfosys.com/ 
Product: http://encompass.endinfosys.com/ 
Demo: 
iLink (Sirsi) Supplier: http:/Awww.sirsi.com/ 
Product: http://www. infobistro.com/ 
Demo: http:/Awww.sirsi.com/Sirsiproducts/elibdemos.html 
iPort (OCLC/PICA) Supplier: http://oclcpica.org/?id=2&In=uk 
Product: http:/Avww.pica.nl/en/news/iport.shtml 
; Demo: http://1 92.87.104.20:51 1 0/iport?language=eng 
MAP (Innovative Supplier: http:/Avww.iii.com/ 
Interfaces) Product: http://www. iii.com/html/products/p_map.shtml 
Demo: 
Metalib (Ex Libris) Supplier: http://www.aleph.co.il/ 
Product: http://www.aleph.co.il/metalib/index.html, http:/Avww-sfxit.com/ 
Demo: 
TalisPrism (Talis) Supplier: http:/Avww.talis.com/ 
Product: http:/Avww.talis.com/pressrele/talisprism.htm 
Demo: 
Xdirectory (Esprit Soutron Supplier: http://www.espritsoutronpartnership.com/ 
Partnership) Product: http://www.espritsoutronpartnership.com/products/xdirectory/index.asp 
Demo: 
ZPORTAL (Fretwell- Supplier: http://www. fdgroup.com/fdi/ 
Downing Informatics) Product: http://www.fdgroup.com/fdi/zportal/about.html 
Demo: 


rest only on compliance with open standards 
rather than proven implementations. To be 
fair it is the closed character of most LMS 
that is the limiting factor. The suspicion with 
late entrants could be that the systems may 
have been launched to protect sales of the 
core LMS; whereas it is clearer that early 
entrants hope for the possibility of creating a 
new market place. This applies particularly to 
Ex Libris, at least for SFX, because of its 
influence on the open URL standard. 


Development paths 

The product development path and partners, 
indeed the commercial model and national 
base of the LMS vendor may themselves 
colour customer perception of the product. 
PICA/OCLC have an impeccable 
background of service to library values and 
OCLC charitable status, and iPort is based on 
work in Decomate (an EC funded project). 
FDi’s products were developed in eLib and 


European projects, most notably Agora. Talis 
is owned by a consortium of UK public and 
academic libraries. These suppliers can speak 
the language of partnership with some 
conviction. Certainly Fretwell-Downing 
marketing is fluent in the language of eLib 
using terms that will be familiar to UK 
academic librarians. Talis rather cheekily talks 
of the Talis Information Environment, 
echoing JISC’s use of the phrase information 
environment to express its vision of an 
information architecture, which again will be 
familiar to university librarians (see 
Appendix). These factors may have an 
importance beyond creating a fluency in 
marketing language. US based suppliers may 
suffer from the fear in customers’ minds that 
systems will not have so clear a fit into UK 
agendas, such as the hybrid library research 
and the requirements of the JISC Information 
Environment and recognition of the 
importance of Athens. Support of Athens is 
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surprisinzly limited in the surveyed products, 
even given that in the next few years Athens is 
due to be replaced by Sparta. Within both 
Metalib and ENCompass Athens compliance 
was reparted only to be a planned future 
development (though SFX is Athens 
compliant). Zportal and MAP claim to be 
compliant. It remains true, however, that the 
core requirements set out by ARL, for 
example, do not notably diverge from those 
set out in the hybrid library literature (ARL, 
2002, Appendix A). It is possible, therefore, 
that the core agenda for libraries 
internationally is much the same. Indeed, 
requirements across library sectors may not 
differ greatly, so that NHS knowledge 
services, public libraries in the People’s 
Network as well as national libraries and 
major business libraries will all require much 
the same systems in the next five to ten years. 


Innovation 

The survey seemed to show that the suppliers 
were being quite slow to take on new 
protocols. For example, the OAI-MPH was 
not supported by most of the systems, though 
the most exciting current developments in 
resource sharing, such as the JISC FAIR 
programme of projects seem to rely on this 
standard (JISC, 2002). This may reflect both 
slowness on the suppliers’ side to develop the 
products, but perhaps more realistically also a 
lack of demand from the library side, given 
that few have as yet bought LPS. Suppliers 
may be more responsive once a critical mass 
of customers emerges. 


Market strategies 

How the products are packaged varies. 
Zportal can be purchased standalone or 
integrated with the Olib LMS and has a 
number of extra “plug-ins” that are bought 
separately (e.g. Z2Web for searching non Z 
compliant Web sites, Zharvest to search Web 
metadata, Z’Mbol for any unstructured data). 
In contrast, Encompass is offered in three 
versions: the full version and two entry level 
products, ENCompass for resource access 
(primarily offering cross searching 
functionality), and ENCompass for digital 
collections (focussed on digitisation). 
LinkfinderPlus, the open linking solution, can 
be bought as part of the three Encompass 
products or standalone. Metalib includes 
SFX as a component that can also be bought 
separately. It also has a digital collections 
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management tool, called Digitool. MAP 
started as part of the core Millennium LMS, 
but has since been productised. It consists of 
three elements: MetaFind (a universal search 
interface), WebBridge (for context sensitive 
resource linking), and Web Access 
Management (for user authentication). 
Whereas most of the systems described here 
are add-ons, Talis PRISM is offered in the 
core library product, at no extra charge. 

Ex Libris supplies with Metalib/SFX a 
KnowledgeBase used to manage linking; as 
does Endeavor with LinkFinderPlus. In the 
Ex Libris model the supplier continues to 
maintain and update the KnowledgeBase for 
client libraries. This may signal a shift in the 
business model for the LMS supplier and 
potentially of the relationship with the client. 
Similarly, there is much more configuration 
work with interoperable systems than with a 
traditional LMS, and this may change the 
business model for suppliers, with much more 
stress on supplying or organising the 
cooperative creation of content and 
customising connections between different 
systems. In other cases this work could be 
undertaken by libraries themselves. 

The ESP offering is more modular than the 
other systems, and rather different from the 
other products discussed here, the primary 
marketing strategy seems to be sell each or a 
number of tools to solve specific problems, 
e.g. offering access to digitised examination 
papers, or to e-journal links, or to distributed 
input into an information catalogue as a 
community information system (Xdirectory), 
or authentication (IxAccess). As a toolkit it 
would require more input to create the 
functionality available out of the box from 
Metalib, say, and some of the others. In fact 
any non-library CMS or portal toolkit could 
be adapted to provide the same functionality 
as such library portal products, but actually 
would involve a lot of work to configure, In 
the UK, Roberts reports on opting not to use 
the most likely ready made library portal 
system for his metadata integration project, 
though the scope of the project is narrowly 
focussed on cross searching of locally digitised 
resources (Roberts, 2002). 

If all this smacks a little of marketing by 
confusion, it may reflect more innocently the 
diverse development paths of the products 
and the suppliers’ struggle to present them in 
an understandable and familiar way to 
potential customers. It also reflects 
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fundamental differences in the products. 
SIRSI’s iBistro and iLink seem to be primarily 
about enrichment of standard bibliographic 
records with cover images, TOCs etc. Metalib 
is built round open linking and SFX, with a 
huge database of resources defined at a fine 
granular level. 


Selling points 

But it remains a confusing marketplace, 
where it is difficult to compare like with like, 
to know which modules will be actually 
required to achieve what is wanted and how 
far modules from different systems could 
really interact. Our survey attempted to shed 
a little light into this confusion by asking 
suppliers to identify what they thought their 
selling points over their rivals were. If nothing 
else, this suggests where the stress in their 
marketing is taking place. The following 
paragraphs represent some of their responses. 

In describing Zpartal FDi stressed support 
from resource discovery through to final 
delivery of the text of a document. Its work an 
complex systems in Australia for ILL puts it 
in a strong place to offer document delivery 
services. But the dominance of the British 
Library as an ILL supplier to UK academic 
libraries may weaken this appeal here. In 
other systems ILL seems to be 
underdeveloped, possibly because a decision 
was made to incorporate the functionality in 
the supplier’s core LMS instead. Zportal also 
stresses that its standard interface was 
designed by an experienced Web designer, 
whereas other suppliers stress openness to 
local customisation. Given the complexity of 
usable Web design and interface design, 
Fretwell-Downing’s identification of this as a 
selling point may be well founded. 

Ex Libris stresses the maturity of Metalib, 
and not just its compliance with standards but 
also the influential nature of SFX and the 
company’s participation in standard 
evolution. The size of the standard SFX 
Knowledgebase (160,000 ISSNs and 40,200 
e-books) is also a strength. It also stresses the 
personalisation tools. 

Endeavor stressed multi-protocol search 
access. It claimed that Encompass was “the 
only product to provide access via Z39.50, 
HTTP, and XML gateways”. Flexibility in 
interface design using XSL was also stressed. 
MAP rather similarly stresses cross searching 
and other linking capabilities and strength of 
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ability of the library to make local 
configuration. 


Linking to learning environments 

For universities, integration of the digital 
library with learning environments is 
increasingly seen as a critical issue. 
ENCompass has stolen a march on its 
competitors with a demonstration at this 
year’s American Library Association summer 
meeting (and at the UK LIS-Show) of an 
ENCompass search screen embedded in the 
learning environment Blackboard 
(Blackboard, 2002). FDi was until recently 
part of the same company as FDE, which was 
itself selling a learning environment, 
especially popular in the FE sector. 
Development projects like EASEL were 
exploring digital library/VLE integration (see 
references for URL). 


Skills required of librarians 

The survey revealed mixed views on what new 
skills librarians might need to manage such 
services. In some cases the suppliers claimed 
that no new knowledge was needed. Others 
thought a working knowledge of XML/XSL 
would be useful (presumably to design search 
result screens). In at least one case, significant 
knowledge of scripting languages, at least to 
support an installation, was recommended. 


Conclusions 


Each library will evaluate the new products in 
relation to the success of existing solutions 
such as those described above and the extent 
of the perceived benefits of integrating 
content for users or enhanced management 
offered by Library Portal Solutions, but 
common evaluation criteria are likely to be: 

* Which LMS is in use locally? Many of the 
products are offered as standalone 
solutions, but the risk of purchasing a 
LPS from a supplier other than the one 
that provides the local LMS is likely to be 
seen as high. 

* Cost, taking into account optional extras 
and configuration by the supplier. 

* Licence terms. The survey revealed a 
great variety of licence models, including 
terms based on the number of staff users, 
size of institution, size of library, number 
of departments using the system. 
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* Basic technology used. Some of the 
systems surveyed were available only on a 
particular flavour of unix or using a 
particular proprietary database. It will 
influence selection if these do not happen 
to be the ones preferred by the institution. 

* Supplier credibility, that is the usual 
measures of a supplier’s long term 
financial viability, such as return on 
capital employed or the Acid test. 

e Search, metadata and content type 
protocols currently supported, relative to 
immediate local requirements. 

* Innovation. Evidence of a quick response 
to the appearance of new protocols, and 
of their listening to customer user groups 
as well as the size of the development 
team and participation in relevant 
standards development (e.g. openURL). 

* Technical competencies of library staff. 

¢ Current and predicted institutional 
initiatives, e.g. university portal or 
standardisation around one VLE. 

e  Pan-institutional initiatives, such as of 
JISC and the development path of its 
Information Environment (see Glossary; 
DNER 2001; Grout 2001). 


The evaluation, purchasing and development 
of library portal solutions is an area that is ripe 
for cross institutional cooperation. This 
article has been written from the perspective 
of academic libraries, but it is likely that 
national, commerical, public and health 
libraries will also purchase such systems. 
International cooperation would also be of 
value. ARL has taken a leading role in 
surveying what libraries are doing and co- 
ordinating thoughts about requirements. The 
underlying library user requirements from 
LPS for many sorts of libraries may not be 
very different. 

Given the complexity and rapid change in 
the systems available, cooperative work in 
defining user requirements and evaluating 
systems would be of great value. For a single 
library to undertake to evaluate all the 
available systems in depth is not practical. 
Furthermore, the very nature of the Web 
based technology in use makes experiments in 
integrating content using various systems 
possible, should the suppliers permit it. At the 
level of procurement, a model request for 
tenders cauld be produced and it would be of 
value to the library community to share 
information about the cost of licences, and 
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licence terms. After implementation the 
sharing of configuration data would greatly 
reduce the load of maintenance of systems in 
use. And if the systems are to be effectively 
used, awareness on the part of data providers 
is necessary and briefing them could be 
another area for cooperative activity. 
Predictions of how the market will develop 
over the next five years are risky, but need to 
be made. Looking at how the LMS market 
has developed long term it is likely that in that 
time frame systems will converge, offering 
very similar functionality and based on the 
same underlying technology. But metadata 
standards, data types and search protocols 
will continue to proliferate and evolve. 
Metalib and some others may become 
established as standalone products, with 
better functionality. But customers will 
probably opt for safety, meaning either 
sticking with their own LMS supplier or 
bandwagoning behind a few products. The 
role of consortial purchasing of systems 
integrating smaller institutions and collections 
with larger ones may also be significant: these 
consortia may not be so dependent on 
incumbent suppliers, creating new market 
opportunities, perhaps internationally. The 
benefits to users in terms of integrated access 
to multiple sources will be great, but not 
revolutionary in improving research. The 
increased access to information about 
patterns of use will make managing 
e-collections much easier; the possibility of 
applying and comparing quite precise 
performance indicators for libraries will be 
increased. But in fact the impact overall on 
the LMS marketplace could well be marginal, 
one or two players will be replaced. The areas 
of difficulty where competition will be 
focussed will be in maintaining larger and 
larger link databases, and above all in the area 
of taxonomic support in multi subject virtual 
collections. Electronic digital rights 
management is another challenge to be 
scaled. It is likely that something will come 
“out of left field” that offers a new vision of 
what needs to be achieved: e.g. powerful 
visualisation at the search interface or the idea 
of creating knowledge communities around 
library portals. However, while LPS already 
offers libraries powerful if expensive tools to 
make it easier for users to access 
heterogeneous information sources, it is 
unlikely that interfaces will become so simple 
that librarians will no longer need to give 
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priority to training users in information skills 
and changing the culture of information use. 
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Appendix. Glossary 


Athens — Eduserv service delivering 
“authentication, authorisation and distributed 
user administration services to UK” HE/FE. 
<http://www.athens.ac.uk/> 

ELib — The UK JISC funded electronic 
library programme whose third phase funded 
research into the hybrid library. 

JISC information environment — JISC’s 
concept of secure and convenient access to a 
comprehensive collection of scholarly and 
educational material. 

Fit to serve all sorts of digital content. 

Fully supporting the submission and 
sharing of research and learning objects, 
providing a range of meaningful, rich and 
innovative methods of accessing electronic 
materials, to enrich and develop the learning 

and research process. 

' A collaborative landscape of service 
providers who work together to seamlessly 
cater for the needs of the community on a 
national basis. 

Underpinned by real world interoperability, 
based upon a common standards framework 
and common semantic for digital resource 
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description and access. <http:// 
www.jisc.ac.uk/dner/development/ 
lEstrategy.html>, also see UKOLN JISC IE 
Architecture Web pages, <http:// 
www.ukoln.ac.uk/distributed-systems/jisc-ie/ 
arch/> 

OAI-MPH -— the Open Archives Initiative’s 
protocol for metadata harvesting <http:// 
www.openarchives.org/> 

Open linking - A NISO supported concept 
for using OpenURLs and link resolvers to 
select and present hyperlinks to appropriate 
related information resources independent of 
the original information resource. <http:// 
library.caltech.edu/openurl/ 

Presentations. htm> 

OpenURL — A metadata transportation 
format, standardising the syntax but not the 
content search queries, planned for NISO 
trial late 2002. <http://library.caltech.edu/ 
openurl/> 

Web services — The term “Web services” has 
a specific meaning, it does not just refer to an 
information service on the Web say. They are 
interfaces for communication between two 
applications, using XML. In the same way 
that programmatic interfaces are available via 
HTML forms, programs are now able to call 
on each other by exchanging XML data 
through an interface, e.g. by using SOAP 
Version 1.2, the XML-based protocol 
produced by the XML Protocol Working 
Group. 

XML is the standard language used by a 
Web service. A Web service will receive XML 
and send the output back as XML. The 
specification provides rigour, structure and a 
standard for importing information from 
location to location. 

“The power of Web services, apart from 
their great interoperability and extensibility 
thanks to the use of XML, is that they can 
then be combined in order to achieve more 
complex operations. Several programs 
providing simple services can interact in order 
to permit complex operations” <http:// 
www.w3.org/2002/ws/> <http:// 
www.ariadne.ac.uk/issue29/gardner/>. 
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Introduction 


Electronic mail (e-mail) is no longer just a 
method of communication, but a way of 
doing business; 94 per cent of businesses have 
access to the Internet and e-mail (Department 
of Trade and Industry, 2001). Individuals are 
becoming more dependent on e-mail 
technologies as a means to transfer and 
receive information. Before computers we 
communicated with each other face to face, 
by telegraph, by letter, by telephone or by fax. 
However, businesses can now send and 
receive e-mails via computers, and this has 
redefined the way in which we communicate 
(Whelan, 2000, p. 2). 

There are several reasons why people and 
businesses choose to communicate more and 
more by e-mail. These include: 

* cost; 
* ease of use; 
* convenience and speed. 


With the seemingly overuse of e-mail this 
paper examines the impact it has on users’ 
work and e-mail load, along with the 
problems of e-mail overload. E-mail overload 
occurs when the number of e-mails being sent 
and received becomes too difficult to manage, 
overwhelming the user. 


Definitions 


We have already defined e-mail overload, but 
two more terms need defining in the context 
of this article. They are e-mail, and the UK 
workplace. 

E-mail is defined as: 


The sending of non-spoken information between 
individuals over a telecommunications network 
to a location where it is stored for subsequent 
retrieval, usually in a computer (New Shorter 
Oxford English Dictionary, 1993). 


The UK workplace implies a variety of 
business and office environments and so is 
defined as: “A variety of offices covering 
different industries in the United Kingdom”. 


Aims 


The main purpose to this study is then to 
establish whether e-mail overload exists in the 
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UK workplace. Studies into e-mail overload 
from the USA have been undertaken (for 
example, Whittaker and Sidner, 1997; 
Goldschmidt ez al., 2001). However, very 
little information exists with regard to e-mail 
overloac in the UK. It is of interest to see 
whether the e-mail overload problem 
experienced, and reported, in the USA is 
beginning to trouble the UK workplace. 
Although there has been much discussion 
regarding information overload and the 
problems associated with it (for example, 

Bawden, 2001; Reuters, 1996, 1997; Lewis, 

1999), e-mail overload in the UK has only 

been discussed in press articles, and these 

often lack detailed analysis. A further 
intention of this study is provide a qualitative 
approach to establishing whether e-mail 
overload exists and to look into how e-mail is 
used in the workplace at a much more 
detailed level. 

The specific objectives of the study were to: 

e Identify how people use e-mail and their 
attitudes towards it and whether position 
in the company is an influence. 

* Establish whether there are any guidelines 
for e-mail usage in the workplace and if 
this is an influencing factor on overload. 

* Devise guidelines on how to effectively 
manage e-mail. 


Literature review 


History of e-mail 

E-mail was first designed as an 
asynchronous[1] communication tool 
(Whittaker ez al., 1997, p. 276; Kimble and 
Abu Bajkar, 2001, p. 371; Adam, 2002, 

p. 88). It has now evolved to more that just a 
communication tool, for example, the inbox 
may act as a task manager rather than just a 
place to store messages. 

E-mail is an evolution of a number of 
previous communication methods, including 
the electronic telegraph, the telephone, the 
telex network and the development of the 
Internet. In fact the first asynchronous e-mail 
message sent between computers took place 
as long ago as 1971 (Campbell, 1998). It is a 
communication tool that is being used more 
and more as it is a quick, cheap and an easy 
way to communicate across time zones. This 
benefit arises from the asynchronous nature 
of e-mail. 
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E-mail characteristics 

Every method of communication has its 

place, however e-mail has proved itself to be a 

strong contender in many situations, 

especially business ones. It is quick and easy 
to use and information can be transmitted 
across the world simply and cheaply. Having 
said this, its strengths can also be its 
weaknesses. For example, it is very easy to 
copy multiple people into a message, but this 
can lead to the transmission of unwanted 
messages that annoy people and, in extreme 
cases, it can lead to system overload. 

As noted earlier, there are a number of 
reasons why people choose to communicate 
via e-mail rather than by other 
communication methods. These reasons are 
also key defining characteristics of e-mail. 
These are summarised below. 

* Cost. E-mail is relatively cheap, and 
information can be sent around the world 
for the same price as a local telephone call 
(Whelan, 2000, p. 4). On the other hand 
there are set up and hidden costs 
associated with e-mail, such as 
connection charges to an e-mail service 
provider, hardware and software for 
security functions and staff to train users 
or administer a helpline. 

e Ease of use. For no extra cost numerous 
people can be copied into the same 
message, whilst reports, spreadsheets, 
presentations and other such information 
can also be included in the message. The 
ability to send files as attachments is a 
huge benefit to businesses, not only 
internally but also externally to clients. 
This can cause problems as the recipient 
may not be able to open the files, it is 
simple to forget to attach a file, it can take 
time to download a large file and viruses 
can be transmitted through attachments. 

* Convenience and speed. One can send and 
receive messages whenever it is convenient, 
anywhere in the world, whilst at the same 
time messages can be sent around the 
world in a matter of minutes. In other 
words, e-mail is available 24 hours a day, 
seven days a week. This is a benefit of 
e-mail being an asynchronous 
communication tool. As quick as it is to 
send a message, some research suggests 
that, on average, it takes 30 minutes to 
write and send a business letter whilst only 
five minutes is spent on an equivalent 
e-mail message (Whelan 2000, p. 27). 
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This suggests that less time is taken over 
writing and thinking about the content cf 
an e-mail, and spelling mistakes are 
nowadays fairly common in business 
e-mails. In a recent study it was found that 
there were 57 spelling mistakes per 100 
e-mails (Mallon and Oppenheim, 2002, 
p. 15). 


E-mail considerations for businesses 
Along with the characteristics associated with 
e-mail, businesses also need to consider 
related issues. These include: 

*  Liability/legal issues. There have been a 
number of legal cases that have 
established that e-mail can be admissible 
as evidence. In a recent survey it was 
found that 53 per cent of UK employees 
have exchanged racist, sexist and other 
discriminatory messages by e-mail. 
Although it may not be the intention to be 
discriminatory or defamatory, employers 
can be liable for it (Stewart, 2002). A key 
case in this area is when Norwich Union 
Insurance Company had to pay £450,000 
plus costs to a competitor after a 
defamatory e-mail was circulated (Nisse, 
1997). 

+ Security issues. Security of e-mail, or lack 
of security, is a key issue for businesses. If 
the incorrect security measures are in 
place then it may be possible for people to 
read or change messages that have been 
sent by someone else. Similarly, it may be 
possible for people to hack[2] into 
another person’s e-mail account and 
read, delete and send messages from that 
account. There are numerous security 
packages on the market and most 
businesses will have adequate security 
measures in place, such as a firewall[3]. 
Software is available to encode and 
transmit messages securely. Braue (2000) 
discusses several options in his article, 
including encryption software. 

* Volume of communication and overload. 
There have been reports of e-mail 
creating information overload. In a survey 
by KPMG Consulting, in 1998, 27 per 
cent of business e-mail users considered 
the current level of e-mails they receive to 
be “excessive” or “intolerable” whilst 41 
per cent believe that in five years’ time the 
level of e-mails they receive will be 
“excessive” or “intolerable” (Whelan, 
2000, p. 8; Dennis, 1998). 
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Information overload 

It is important to understand the concept of 
information overload before discussing that of 
e-mail overload. There is no single definition 
of information overload, it has also been 
called simply too much information, info glut, 
or as Shenk (1997) puts it, data smog. A final 
term found in the literature, which is used to 
describe the problem of information overload, 
is information fatigue syndrome (Oppenheim, 
1997, p. 246; Lewis, 1999, pp. 3-4). 

The definition given by Bawden (2001) is 
the one closely linked to that of e-mail 
overload and is the definition used in this 
paper: 

Information overload occurs when information 


received becomes a hindrance rather than a help 
when the information is potentially useful. 


Research has found that information overload 
can lead to stress, loss of job satisfaction and 
physical ill health (Lewis, 1999, p. 4). 


E-mail overload 

No formal definition of e-mail overload has 

been found in the literature, although Hiltz 

and Turoff (1985, p. 682) describe what 

information overload means in the context of 

computer-mediated communications systems 

(CMCS)[4]. They say that in the context of 

CMCSs the term information overload refers 

first to the delivery of too many 

communications and to an increase in social 

density that gives individuals access to more 

communications than they can easily respond 

to. They also use the term information 

entropy, which means that the incoming 

messages are not sufficiently organised by 

topic or content to be easily recognised as 

important. As a consequence they suggest 

that individuals might: 

* fail to respond; 

* respond less accurately than they would 
otherwise; 

* — respond incorrectly; 

* systematically ignore (i.e. filter) some 
features; 

* recode to a more compact or effective 
form; or 

* quit (in serious cases). 


Ferris Research (Ferris, 2000) established 
that corporate e-mail users spent five minutes 
a day dealing with each e-mail. With the 
average number of e-mails received a day in 
the UK being 39 (Pitney Bowes, 2000), on 
average, over three hours a day is spent 
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dealing with e-mail. By comparison, recent 
research by the Department of Trade and 
Industry found that workers took an average 
of 49 minutes a day to sort out their inboxes 
(Lyons, 2002). The research does not 
mention whether this included replying to 
messages, which is implied when “dealing 
with e-mails” but not when “sorting out an 
inbox”. 

A study by the Chartered Management 
Institute (2001) found that 32 per cent of 
managers suffered from a deluge of e-mails. 
In another study from the Chartered 
Management Institute it was also found that 
the need to keep up with e-mails was ranked 
in tenth place in a league table of work 
stresses along with interruptions, deadlines 
and getting the right information. It was 
ranked higher than having a bad relationship, 
working environment or customer complaints 
as a cause of pressure and stress (BBC, 2000). 
With information being increasingly sent to 
individuals via e-mail, it raised the question of 
whether it is just managers it affects. 

In a recent MORI survey of 200 UK 
managers and directors it was found that a 
quarter send over 30 e-mails a day, with a 
further 40 per cent sending between 11 and 
30. It was also found that more that half the 
directors receive more that 20 messages a day 
(Flood, 2001). 


E-mail overload solutions 
Several recommendations on how to tackle 
e-mail overload have been made. Whittaker 
and Sidner (1997) recommended that e-mail 
systems should be redesigned to support filing 
and task management as well as producing 
asynchronous communication. This would 
mean that important messages were not lost 
and unanswered messages would not pile up. 
Kimble et al. (2001, p. 371) suggested that 
the sender-side of e-mail communication 
should be focused on rather than the 
recipient-side. In earlier studies it has been 
established that one source of e-mail overload 
could be the misuse of the cc (carbon copy) 
facility found in e-mail systems (Kimble et al., 
1998, p. 411) and that a way to reduce e-mail 
overload would be to remove the cc facility 
(Kimble and Abu Bakar, 2001, p. 373). In the 
2001 paper they described a prototype system 
where the sender could tag to whom each part 
of a message should be sent, and the recipient 
then only received the part that was applicable 
to them. In testing it was found that tagging 
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was cumbersome and users would have to be 
familiar with HTML. At the same time the 
evaluators felt that time could be saved in a 
business situation and that its use could result 
in the reduction in the volume of e-mail 
received. 

In reviewing the literature it has been found 
that solutions to e-mail overload have tended 
to be system based rather than user based. 
The most notable user studies have been 
those of Mandell (2001, p. 45), Sherwood 
(n.d.), and Financial Mail on Sunday (2000). 
They suggest: 

* The establishment of a corporate e-mail 
policy. 

* Using message headers to help decide 
what to, or not to, read. 

e Marking urgent messages in the header 
field. 

+ Dealing with e-mails in order of priority. 

+ Setting aside time to review the day’s 
messages. 

* Setting up folders to file items away. 

* Developing the habit of only handling 
messages once, deleting or filing them. 


E-mail has developed rapidly in recent years, 
and technological developments (and 
popularity) suggest it may continue to 
develop and grow in the future. It has become 
a vital business communication tool due to it 
being fast, cheap and easy to use, a feature 
largely due to its asynchronous nature. 
However, ease of use makes it possible for 
communications and information to be 
spread far wider than has previously been 
possible, leading to the inevitable concerns 
about overload. The existence of e-mail 
overload appears to be largely anecdotal at the 
moment, suggesting that there is significant 
scope for this study to establish whether the 
phenomenon really exists. 


Methodology 


The methodology employed for this study 
involved the collection of qualitative and 
quantitative material through a series of in- 
depth interviews with 20 subjects. 
Quantitative data were collected from the 
interviewees in the form of a diary counting 


‘the number of e-mails sent and received in a 


week. 
A cross section of interviewees were 
selected, differing in age, position within 
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companies and industry in which they work. 

Selection started with personal contacts of the 

writer, which then snowballed as interviewees 

suggested other possible interviewees. It was 
decided to obtain a wide range of interviewees 
to see if any comparisons can been drawn 
between various groups of e-mail users. For 
example, does seniority or position within an 
organisation impact on a user’s e-mail load? 
The interviews were semi-structured, 
allowing scope for discussion. The interview 
schedule was adapted from that used by 

Whittaker et al. (1997, p 280) in their 

research into e-mail overload in the USA and 

consisted of the following parts: 

* Section one dealt with generalities and 
demographic information. The 
interviewee was given the option to 
remain anonymous, but asked to provide 
age, gender and job title. 

e Section two sought to establish how long 
the interviewee had been using e-mail and 
what they used it for. 

e Section three questioned how the 
interviewee uses e-mail at work and their 
attitudes towards it. 

¢ Section four enquired about their 
correspondence management, how they 
use the cc (carbon copy) and blind 
carbon copy (bcc) functions. 

* Section five collected quantitative data 
regarding how many e-mails they sent 
and received. 

* Section six enquired about their reading 
and reply strategies. 

+ Section seven asked about filing 
behaviour and collected more 
quantitative data about how many e-mails 
the interviewee had in their inbox and 
filing system. 


It has been argued that one can be more 
confident in the validity of the findings if you 
take the interview findings back to the 
participant, otherwise known as respondent 
validation (Silverman, 2001, p. 233). 
Because of this a summary of the interview 
was sent to the interviewee shortly after the 
interview had taken place, along with the 
statistics they provided from their diary. This 
enabled them to comment on the findings 
presented to them before they were used in 
the paper. It also gave them the opportunity 
to furnish further comments that they may 
have not mentioned at the time of 

the interview. 
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Results and findings 


The results of this study are reported in 
accordance with the interview schedule 
discussed above. They are supplemented by 
the results from the diaries of e-mails sent and 
received over a one-week period. 


General e-mail questions 

Table I shows how long each interviewee has 
been using e-mail in a work situation. A top 
left to bottom right trend in the number of 
years spent using e-mail, i.e. those who have 
used e-mail for the shortest period of time to 
be younger than those who have used e-mail 
for the longest period of time, would be 
expected. However this was not the case. 

Those who were over 40 had been using 
e-mail for the same amount of time as those 
under 40. This says something about the 
recentness of the take-up of e-mail and its 
penetration into organisations as a business 
communication tool. This also implies that 
those in the 18-29 age bracket have been 
using e-mail nearly all their working lives. 
Only one interviewee has used e-mail for over 
ten years, but he is in the information 
technology and computing industry so this is 
to be expected. 

From Table I it is possible to work out the 
average length of time an interviewee has used 
e-mail in a work situation. By rounding up the 
figures to the highest number in each category 
(i.e. assuming the interviewee has been using 
e-mail for six years if in the 5 to 6 category 
and that those in the over ten category have 
been using it for 15 years), then the average 
time an interviewee has used e-mail is 5.7 
years. 

E-mail is used for a variety of business 
purposes including: answering questions, 
arranging meetings, keeping up to date, 
networking, sharing ideas, social and general 
work correspondence. As well as general 


Table | E-mail usage in the workplace 


Age 


Years using e-mail 18-29 30-39 40-49 50 plus 


Under 1 
1 to 2 
3 to 4 
5 to 6 
7 to 8 2 1 
9 to 10 1 
Over 10 1 


N N Ww “a 
= NN — 
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communication, e-mail is used for sending 
other sorts of information, and the various 
uses stated by the interviewees are given in 
Figure 1. 

As can be seen from Figure 2, the most 
popular reason for using e-mail is the 
convenience and speed that it offers. Auditing 
purposes, when you receive a message telling 
you if the recipient has read the message or 
not, and having a written record of what was 
said were also popular reasons. 


Attitudes to e-mail 

A total of 65 per cent of interviewees said that 
the vast majority of their work 
correspondence was sent via e-mail, with the 
remaining 35 per cent saying that most of 
their correspondence was sent via e-mail. 
These are significant statistics as one can see 
that e-mail has become the preferred form of 
communication in the business environment. 
Of those who said the vast majority of their 
correspondence was sent via e-mail, 46 per 
cent said that e-mail was by far the most 
dominant communication tool they used. 


Figure 1 Information sent and received via e-mail 
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The majority of my correspondence is sent by 
e-mail. In fact my correspondence is almost 
exclusively sent by e-mail. I have been in my 
recent role for nearly three years and wrote my 
first letter a couple of weeks ago — I did not even 
know which paper to use to print it out on 
(Interview ten). 


Of the interviewees, 50 per cent classed e-mail 
as a “very useful communication tool”, with 
the other 50 per cent saying it was just 
“useful”. However, e-mail does have its 
problems, with 30 per cent mentioning the 
amount of spam or junk mail received. None 
of the 30 per cent had spam filters in place, 
whereas two other interviewees mentioned 
that they had spam filters in place and so were 
not so affected. 

The interviewees were also asked which 
other communication tools they used. All 
mentioned the telephone (100 per cent). 
Figure 3 shows what other forms of 
communication were used, excluding the 
telephone. It is interesting to note that only one 
person mentioned using a mobile telephone as 
an alternative form of business 


What information are you sending and receiving via e-mail? 
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Figure 2 Why e-mail is used over other communication methods 
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Figure 3 Other forms of communication used, other than e-mail 
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communication. Especially so, as it has been 
found that 86 per cent of businesses use mobile 
telephones (Whelan, 2000, p. 44). This could 
be because the mobile telephone is becoming 
synonymous with the traditional telephone. 
The interviewees were also asked if e-mail 
had changed their working life in any way. 
Seven (35 per cent) of the interviewees said 
that e-mail had not changed the way they 
work as they had always had e-mail. The 
results from the other 13 interviewees (65 per 
cent) are in Table II, note that the 
interviewees were not limited to one answer. 


Correspondence management 

The next section in the interview examined 
the interviewees’ use of the cc and bece 
functions. It has been suggested by Kimble 
et al. (1998, p. 411) that a way to reduce 
e-mail overload would be to remove the cc 
facility and so it was important to establish if 
the use of cc and bcc impacted on the 
interviewees’ work and e-mail load. 


Table Il How e-mail has changed the way people work 


Has e-mail changed the way you work? Number 


Communication not as location specific 1 
Created a workload that was not there 
before 
Do not send as many faxes 
Do not use the telephone as often 
Do not write as many reports or memos 
Do not write or send as many letters 
Has changed — but not in a quantifiable 
way 1 
Not as much face-to-face communication 2 
Not so paper based — everything now 
sent electronically 1 
Pattern of the working day has changed 1 


ann Ww a 


All interviewees (100 per cent) used the cc 
function in their work correspondence, 
although 25 per cent of them either only used 
the feature occasionally or very occasionally. 
There were a number of reasons given for 
using cc. The main reason for interviewees at 
staff level was to keep managers or other 
members of the team informed of what they 
were doing. For those at a more senior level 
the main reason to use the tool was to invite 
comments from other team members or other 
departments. 
I copy my boss into an e-mail as it is an easy way 
of keeping him up to date with progress on a 
project (Interview two). 
I use the cc feature as a means to inform those 
people who need to know of proposals, 
suggestions, developments and to invite their 
comments as appropriate (Interview eleven). 


The number of interviewees who used the bcc 
feature was much less than the cc. Only 30 
per cent of the interviewees said they used the 
bcc feature and then it was only on a very 
occasional basis. One interviewee uses it to 
keep managers informed, which was similar to 
his reason for using cc. Another used it to 
send messages to her home account to enable 
her to work on something at home. The 
reasons behind not using bcc were that it is a 
“devious” tool and that the other addressees 
should be aware to whom the message was 
sent. It is all about transparency. 

Another consideration, when asking the 
interviewee about their cc and bec use, was to 
establish whether or not other people within 
the workplace used the function and whether 
or not it affected their e-mail load. A total of 
55 per cent of the interviewees said that the 
way other people used cc and bee did affect 
their e-mail load, whilst 45 per cent said that 
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it did not. The responses are very evenly split, 
however, when examining the data by level of 
seniority we see a difference. The three levels 
of seniority used in this study are staff, middle 
management and senior management. 

Figure 4 shows how the use of cc and bec 
increases the e-mail load as the level of 
seniority increases. The most senior are 
affected more by the use of cc or bcc than any 
other level, and those at staff level are the least 
affected. 

This follows on from why people use the 
two features. As mentioned above, the main 
reason for using the features at a staff level 
was to keep managers informed of how work 
on a project or task was progressing. This 
would imply that managers are therefore 
receiving more messages that have been 
copied or blind copied to them for 
information, which in turn will affect their 
e-mail lead. At a more senior level messages 
were being copied to other staff members for 
comment and therefore not necessarily being 
copied down to lower staff members. This 
implies that they are not receiving as many 
messages and therefore are not as affected. 


Volume of e-mails sent and received 
The next task in the interview was to establish 
how many e-mails were sent and received in a 
one-week period. These data are summarised 
in Table III. 

From these data we can establish the mean 
number of e-mails sent and received from the 
interview group and also for each seniority 
level. The mean gives the average number of 
e-mails sent and received, and is calculated by 
summing the set of data values and dividing 
by the number of values. From the mean it is 
then possible to work out the standard 
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Table Ill Number of e-mails sent and received by 
interviewee 


Level Interviewee No. Received Sent 
Staff 1 96 17 
Middle 2 80 100 
Staff 3 65 60 
Middle 4 45 28 
Staff 5 250 250 
Middle 6 74 57 
Middle 7 52 22 
Middle 8 222 107 
Staff 9 30 9 
Senior 10 130 39 
Senior 11 62 50 
Senior 12 60 19 
Senior 13 25 20 
Senior 14 87 98 
Senior 15 350 250 
Senior 16 89 31 
Middle 17 110 68 
Senior 18 17 12 
Senior 19 100 82 
Senior 20 600 450 


deviation. This measures the spread, or 
dispersion, of the data around the mean. A 
high standard deviation implies a greater 
difference between cases. The results of these 
two calculations are shown in Table IV. The 
top calculation shows the data set as a whole, 
with the results for each of the calculations 
broken down to level of seniority. 

It can be seen from Table IV that the mean 
number of e-mails sent was 87 a week, or 17 a 
day. The mean number of e-mails received a 
week was 127, or 25 a day. The figure of 
receiving 25 e-mails a day is much lower than 
that given by Pitney Bowes (2000), which was 
given as 39 a day a few years back. Staff 


Figure 4 Level of seniority and effects of e-mail load using cc and bec 
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Table IV Mean and standard deviation of e-mails sent and received 


By level of seniority 
Overall Staff Middle Senior 





Mean of sent items 87 77 64 105 
Standard deviation of sent items 107 98 35 140 
Mean of received items 127 110 97 152 


Standard deviation of received items 139 97 65 140 





receive 22 messages a day, middle 
management 19 and senior management 30, 
all of which are lower than the figure quoted 
by Pitney Bowes. This, however, conflicts 
with data from Xerox, who found that UK 
workers received 28 e-mails a day, which is 
more in line with the results presented here 
(Sales Promotion, 2001). 

As mentioned earlier, the standard deviation 
measures the spread of the data from the 
mean. The lower the standard deviation the 
more close together the data. One can see 
from the calculations in Table IV that the data 
from all levels of seniority is fairly well spread. 
This can also be seen in Table III where one 
senior management interviewee received 
approximately 20 e-mails a week and another 
600. The calculations show that the data given 
for middle management is most closely 
grouped, with that for senior management 
being the most diffuse. 

From the diary data it was also possible to 
work out the average number of non-work 
related e-mails sent and received. On average, 
21 non-work related e-mails are received on a 
weekly basis (or four a day), whilst seven non- 
work related e-mails are sent (just over one a 
day). This shows that using e-mail for a 
personal use whilst at work does not adversely 
impact on the number of e-mails sent and 
received. 


Prioritisation, reading and reply 
strategies 

The interview then moved to how long the 
interviewees spent dealing with e-mails on a 
daily basis. Figure 5 shows this graphically. As 
can be seen from the graph, the most time 
spent dealing with e-mails is between three 
and four hours, with the majority of 
interviewees spending up to an hour daily. By 
rounding up the figures to the highest number 
in each category, i.e. assume that an hour is 
spent dealing with e-mails if in the 30 minutes 
to an hour category, then the average time 
spent dealing with e-mails is approximately 
one hour 40 minutes a day. 
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In 2000 Ferris Research (Ferris, 2000) 
established that corporate e-mail users spent 
five minutes a day processing each e-mail. 
The question asked in this study was how 
long an interviewee spent dealing with 
e-mails, not simply reading e-mails coming in. 
Dealing with implies both sending and 
receiving. Therefore one needs to include 
e-mails that have been sent as well as received 
when calculating the average time. The 
average number of e-mails sent and received 
is 42. In which case an average user spends 
2% minutes dealing with e-mails in general. 

However, the calculation should only use 
the number of e-mails received as, in the 
majority of cases, sending an e-mail is part of 
dealing with a received e-mail. Therefore the 
time spent dealing with e-mails is probably 
closer to 4 minutes than 22 minutes. This is 
the view taken in this paper. 

It is difficult to compare the figures 
presented here with those quoted by Ferris 
Research, as it is not entirely clear whether 
five minutes relates to sending and receiving 
e-mails, just sending or just receiving. 

Taking this down to the level of seniority, as 
in Figure 6, one can see that it is the senior 
managers that spend the most time dealing 
with e-mails. This is in line with the mean 
number of sent and received e-mails — with 
senior managers sending and receiving the 
most e-mails. 

Staff spent the least amount of time dealing 
with e-mails, but there is a variation in the 
data as one interviewee at staff level spent four 
hours dealing with e-mails daily and two 
others under 30 minutes. 

Interviewees were asked if they felt the 
volume of e-mails affected their decision 
making. The reason behind this was two-fold. 
First, in the Reuters (1996) report Dying for 
Information it had been found that nearly half 
the managers questioned believed that 
important decisions were delayed and 
adversely affected as a result of having too 
much information. Second, it gave an insight 
into whether or not a user was overwhelmed 
by the number of e-mails they had and were 
therefore, by definition, overloaded. 

Of the 20 interviewees, two felt that they 
could not answer the question, the percentages 
given are therefore based on 18 respondents. A 
total of 50 per cent felt that e-mail did not 
affect their decision making process in any 
way, with 50 per cent admitting that it did. Of 
the 50 per cent whose decision-making 
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Figure 5 Time spent dealing with e-mails daily 
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Figure 6 Time spent dealing with e-mails by level of seniority 
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processes were affected, 56 per cent said it was 
in a helpful way with 44 per cent saying it 
affected them adversely. 

Table V provides an analysis by level of 
seniority and the decision-making process. 
One can see that it is middle and senior 
management who are most affected by 
e-mails, with 60 per cent of senior 
management and 50 per cent of middle 
management saying e-mail did affect their 
decision making. No interviewees at staff level 
found e-mail affected their decision making. 

The two quotes below illustrate how e-mail 
can affect decision making: 


Table V How e-mail affects the decision-making process 
by level of seniority 


Staff Middle Senior 


Yes — it is a help 0 2 3 
Yes — it is a hindrance 

No - it does not affect me 
No answer 


oO A Oo 


1 3 
2 3 
1 1 


Sometimes I find I am not as thorough as I might 
have been, as I feel under pressure to reply to the 
e-mail quickly (Interview two). 

It is better as a consequence of what I can find 
out from it, I can gather intelligence through it 
(Interview fifteen). 


As shown in Table V, four interviewees said 
e-mail was a hindrance rather than a help. Of 
these, two felt on top of the number of e-mails 
they sent and received, however, the decisions 
they were making had to be made more 
quickly than they would have liked as they 
had to move on and deal with the next e-mail. 
The other two interviewees showed signs of 
overload: 


E-mails are essential to my decision-making 
process, but when you’ve got lots of e-mails just 
to deal with them means that you make faster, 
less thought out decisions than you might do 
otherwise. Things start to get out of hand when 
there are a lot of unread messages interspersed 
with read ones. It’s usually when I’m very busy 
and I have to be selective about what I read. But 
then I have to find the time to go through the rest 
... (Interview ten). 
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E-mails are disruptive to the decision-making 
process. If I start to get more than 30 e-mails in 
my inbox then I start to get anxious. E-mail is 
definitely a stressful medium (Interview 
nineteen). 


Interviewee 19 mentioned a new symptom 
that influences the feeling of being 
overwhelmed and hence overloaded — 
information anxiety. Wurman (1989) first 
introduced the term information anxiety in 
his 1989 book entitled Information Anxiety. 
Since then there have been several articles 
relating to it. Information anxiety can have 
many forms, but the main one, and the one 
shown here, is the frustration that arises from 
the inability to keep up with the amount of 
data present in life (Shedroff, 2000, p. 220). 

This suggests that e-mail anxiety is a 
condition associated with e-mail overload. It 
is defined by Brown et al. (2002, p. 7) as: 


The fear or apprehension associated with real or 
anticipated use of a computer to communicate 
[via e-mail]. 


Based on the definition from Shedroff (2000) 
for information anxiety and Brown et al. 
(2002) we can define e-mail anxiety as: 

A fear, sense of apprehension or frustration 


associated with the inability to keep up with the 
number of e-mails on a system. 


With only two interviewees showing signs of 
overload, the question “is there a time when 
you do not feel on top of your e-mails” was 
asked to ascertain if e-mail can be 
overwhelming at times. Thirteen interviewees 
responded to this question, and of those, 70 
per cent felt overwhelmed by the number of 
e-mails they had waiting on their system if 
they had been out of the office for a couple of 
days. So whilst only two exhibited symptoms 
of overload, 13 of the interviewees were aware 


Table VI How e-mail can get out of hand 


Reason Number 


Out of the office for a couple of days 9 
Number of e-mails received from mailing lists 

Not being able to clear the inbox at the end of the week 
Lots of unread messages in the inbox 


N= = 





Table VII Mean number of e-mails on system by level of seniority 


By level of seniority 
Overall Staff Middle Senior 


Mean number of e-mails in inbox 797 180 165 1,493 
Mean number of files 19 16 15 23 
Mean number of e-mails in files 2,058 489 421 3,866 
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that e-mail could get out of hand and could be 
difficult to control at certain times. Table VI 
shows the reasons why e-mails could get out 
of hand. 


Filing behaviours 

Further to the number of e-mails sent and 
received, data were also collected relating to 
the number of e-mails in the inbox, the 
number of files the user had and the number 
of e-mails kept in these files. This is 
summarised in Table VII. 

From Table VII we can see that the mean 
number of e-mails in the inbox was 797 and 
the mean number of e-mails filed was 2,058. 
This is the exact opposite to what was found in 
a US study in which it was established that the 
mean number of inbox items was 2,482, with 
the mean number of e-mails filed standing at 
858 (Whittaker et al., 1997, p. 281). 


E-mail policies and guidelines 

The interviewees were also asked if their 
companies had an e-mail policy, or basic e-mail 
guidelines, and whether this affected their filing 
behaviour. Often companies will place a limit 
on the number of e-mails an employee may 
have, so as not to slow up the system. Only 45 
per cent of the interviewees said that their 
company had an e-mail policy or e-mail 
guidelines, 55 per cent said that there was none. 
This response was very surprising, especially as 
there are legal issues surrounding e-mail, such 
as being used as evidence in court. However, 
recent research has found that only 25-30 per 
cent of businesses do have an e-mail policy 
(Clearswift, 2002). 

One aim of this project was ta devise 
guidelines on how to effectively manage e-mail. 
These do not cover the legal aspects of e-mail 
use, such as monitoring or company disclaimers 
as this was outside of the study’s remit. Based 
on points raised in the literature review and 
during the interviews, the following guidelines 
have been developed. They have been devised 
with reference to Mandell (2001, p. 45), Ferris 
(2000), Hogg (2000), Sherwood (n.d) and 
Financial Mail on Sunday (2000). It was also 
possible to refer to a set of “e-mail golden 
rules”, which were mentioned by interviewee 
two, when devising these guidelines. 


E-mail guidelines and etiquette 
(1) Choose the right communication method: 
* E-mail is not always the best 
communication tool to use. 
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* Do not forget that other methods can 
be better at times. Talking to 
someone or picking up the telephone 
may often save time and 
misunderstandings. 

(2) Make messages clear: 

* Give e-mails informative subject 
headings to enable people to organise 
and prioritise work. Explain why the 
reader has your message and what is 
needed from them. 

* Summarise key issues of an 
attachment in the main body of the 
e-mail. 

* Think whether the attachment really 
needs to be sent. Set up shared drives 
for documents and information used 
as a department or team. 


(3) Style: 
* Write your message in clear, plain 
language: 


— check your spelling and grammar 
before sending a message; 

— avoid sending very long messages. 

* Be polite, employees have been 
sacked for sending e-mails of a 
defamatory nature. 

(4) Addressing: 

* Do not over use the cc or bec 
functions — remember their usage 
affects other people’s e-mail load. 
Think: does this person really need 
this message before including them? 

* Do not automatically “reply to all”. 

* Check that you have chosen the 
correct address before sending a 
message. 

(5) Dealing with incoming e-mails: 

* Delete junk mail immediately and set 
up filters to stop them entering inbox. 

* Read e-mails on a daily basis. Deal 
with them in order of priority 

+ Respond to important messages 
immediately. 

+ Try to handle messages only once. 
Then either delete or file 
immediately. 

+ Be selective with the number of 
mailing lists subscribed to. Are they a 
necessary part of the job? 

* On holiday? 

— set up an out of office reply so the 
sender knows when you will be 
back; 

— give another contact name who may 
be able to help in your absence. 
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(6) Setting deadlines via e-mail: 
e Be realistic, explain why such a 
deadline has been set, and remember: 
— colleagues have other priorities; 
— not everyone works full-time. 
e Mark urgent messages clearly in the 
subject header. 


Conclusions and recommendations 


E-mail in the workplace is used for a variety of 
purposes, including arranging meetings, 
networking and keeping-up-to-date. E-mail 
comes into its own when sending and 
receiving information and it is incredibly 
useful for sending around documents and 
attachments. 

All this is reinforced by the finding that 65 
per cent of interviewees said that the vast 
majority of their work correspondence was 
sent via e-mail, with the remaining 35 per 
cent saying they used e-mail most of the time 
for work correspondence. With the average 
length of time the interviewees have had 
e-mail being 5.7 years, and 65 per cent saying 
that the vast majority of their work 
correspondence was sent via e-mail, then 
e-mail has had a very significant impact, as a 
business communication tool, in a very short 
space of time. 

E-mail has changed the way a lot of people 
work, however, seven (35 per cent) of the 
interviewees have had e-mail all their working 
lives and felt that it had not changed the way 
they worked in any way. Mostly the 
interviewees found that they did not use the 
telephone as often or did not write as many 
letters as they used to. This is also reflected in 
the figures above with 100 per cent of the 
interviewees using e-mail most of the time to 
communicate in a business environment. 
With 86 per cent of businesses using mobile 
telephones (Whelan, 2000, p. 44) it was 
surprising that only one interviewee 
mentioned this as an alternative 
communication tool. This could be because 
the mobile telephone is considered to be the 
telephone. 

It has been suggested by Kimble et al. 
(1998, p. 411) that to reduce e-mail overload 
one should remove the cc and bcc facilities. 
Of the senior managers interviewed, 80 per 
cent found that the use of the ce and bec 
function by other work colleagues did impact 
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on their e-mail load, and that they received 
more e-mails as a result. 

Senior management, on average, receive 
152 e-mails a week, or 30 a day; whereas, on 
average, staff receive 110 e-mails a week and 
middle management 97. Similarly, senior 
management send more e-mails. On average, 
senior management sent 105 e-mails a week, 
with staff sending 77 and middle management 
64. This explains why senior management 
complain of the use of the cc and bcc 
functions more than those at a lower level. 

Whether the cc and bec functions should be 
removed or not is a different matter. All 
interviewees said that they were a useful tool 
to have. They are used primarily to keep 
people informed of what is happening in the 
company and to collect comments about a 
certain issue from other colleagues. 
Guidelines in their usage may be more 
effective than removing the facility altogether. 

This study has found that the average time 
spent dealing with e-mails is approximately 1 
hour 40 minutes a day. Using this and the fact 
that the average number of e-mails received 
daily is 25, then it takes about four minutes to 
deal with each e-mail. It is assumed here that, 
in the majority of cases, sending an e-mail is 
part of dealing with a received e-mail and so 
the figures for sent e-mails need not be 
included in the calculation. Again it is senior 
management who spend the most time over 
e-mails, which is in line with the fact they 
send and receive more of them. 

On a weekly basis, an average of 21 non- 
work related messages were received and 
seven sent weekly. Therefore personal, non- 
work related messages do not adversely affect 
the overall number of e-mails being sent and 
received. 

The interviewees were also asked about 
their filing behaviour. It was found that the 
average number of e-mails in an inbox was 
797. The average number of files, other than 
the inbox, stood at 19, with the average 
number of e-mails in files being 2,058. As can 
be seen, the average user has a significant 
number of e-mails and information to deal 
with. 

Nine out of 13 interviewees (70 per cent) 
were aware that e-mail got out of hand and 
was difficult to control at certain times. The 
reasons behind this included being out of the 
office for a couple of days, the number of 
e-mails received from mailing lists and having 
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lots of unread messages amongst the read 
messages. 

Of the 13, two interviewees showed clear 
signs of overload. One interviewee felt the 
number of e-mails he had to deal with was 
overwhelming, whilst the other mentioned 
feeling anxious when the number of e-mails in 
his inbox went above 30. Information anxiety 
was not a symptom originally identified as a 
sign of overload but is certainly one to be 
taken into consideration when identifying 
overloaded e-mail users. It also suggests that 
e-mail anxiety, rather than information 
anxiety, should be a condition associated with 
e-mail overload. 

_ Hiltz and Turoff (1985, p. 681) found that 
feelings of overload would generally reach a 
peak in the early stages of using e-mail, when 
users had gained enough familiarity to cause a 
large number of messages to arrive, but not 
enough to know how to deal with them. This 
is not in line with this study. Here the two 
interviewees showing signs of overload in this 
study have been using e-mail for six and nine 
years respectively. 

Only 45 per cent of the interviewees said 
that their company had an e-mail policy, with 
one of these also mentioning a set of “golden 
e-mail rules”. This is surprising due to the 
legalities associated with e-mail. 

As a result of this study a set of e-mail 
guidelines have been devised. It is hoped that 
these will help users to effectively manage 
their e-mail load. 

Whilst it has been established that e-mail 
overload exists, it is difficult to say to what 
extent it exists as data were only taken from a 
very small number of people and 
organisations. E-mail overload is a very 
personal experience. One person may receive 
200 messages a week and not feel overloaded, 
whilst another may also receive 200 messages 
a week and feel overwhelmed by them and 
hence overloaded. As e-mail is such a 
personal medium, guidelines are the most 
appropriate way of helping deal with the 
overload problem, as they can be adapted for 
personal use. 

A logical step from this study would be to 
examine the extent of e-mail overload in 
separate industries rather than the workplace 
as a whole. Similarly, a detailed case study of 
one organisation would also be beneficial, 
with further research into how an organisation 
communicates with its staff. This would also 
enable more detailed analysis into how e-mail 
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is used in the workplace and the extent to 
which e-mail overload affects employees. 

Further recommendations for research 
include: 

e Research into junk mail and the extent it 
affects e-mail users, 

°- The impact of mobile telephones in 
organisations. 

* Research into the effect of introducing 
e-mail guidelines to an organisation and 
whether the number of e-mails sent and 
received changes over a period of time. 

* Looking into how widespread e-mail 
policies are in the workplace and what the 
legal implications are for not having one, 
there is an opportunity here to look at a 
specific industry. 


Notes 


1 Asynchronous means that the sender and receiver 
of a message do not need to be online at the same 
time because messages are stored until retrieved by 
the recipient. 

2 Hacking is when someone gains unauthorised 
access to computer files or information held on a 
computer (The New Shorter Oxford English 
Dictionary, 1993). 

3 A firewall is a piece of hardware or software which 
performs a screening process on communications 
from networks external to organisations, with the 
aim of blocking unauthorised access while 
permitting outgoing communication (Oxford English 
Dictionary Online, 2001). 

4 Computer-mediated communication systems use 
computers and telecommunications networks to 
compose, store, deliver and process communication, 
this includes e-mail, computerised conferencing and 
bulletin board systems (Hiltz et al., 1985, p. 680). 


References 


Adam, R. (2002), “Is e-mail addictive?”, Aslib 
Proceedings, Vol. 54 No. 2, pp. 85-94. 

Bawden, D. (2001), “Information overload”, Library and 
Infermation Briefings Series, No. 92, LITC, 

South Bank University, London. 

BBC (2000), E-mail Stress Overload, news.bbc.co.uk/1/hif 
uk/654956.stm (accessed 19 August 2002). 

Braue, D. (2000), Securing E-mail in an Insecure World, 
available at: www.apcmag.com/apc/apcmag.nsff 
{_alldocs/BCB2D52E9D469FCBCA256966001 DOBE1 
{accessed 11 September 2002). 

Brown, S., Fuller, V. and Vician, C. (2002), individual 
Characteristics and E-learning: The Role of the 
Computer, Anxiety and Communication 
Apprehension, available at: www.kelley.iu.edu/ 
ardennis/wp/tr122-1.doc (accessed 11 September 
2002). 


Aslib Proceedings 
Volume 55 > Number 3 : 2003 - 166-180 





Campbell, T. (1998), The First E-mail Message, available 
at: www.pretext.com/mar98/features/story2.htm 
(accessed 11 September 2002). 

Chartered Management Institute (2001), Workplace 
Revolution Jolts UK Managers, February, available 
at: www.managers.org.uk/press/revolution.html 
(accessed 22 July 2002). 

Clearswift (2002), IDC's ‘Policy-based Content Security’ 
White Paper Stresses Increased Awareness and 
Importance of Content Security, available at: 
www.clearswift.com/news/pressreleases/items/ 
109.asp (accessed 31 August 2002). 

Dennis, S. (1998), “Intranets equal info overload claims 
KPMG", Newsbytes, 14 April. 

Department of Trade and Industry (2001), Business in the 
Information Age: International Benchmarking Study, 
available at: www.ukonlineforbusiness.gov.uk/ 
main/resources/publication-htm/bench2001.htm 
(accessed 11 September 2002). 

Ferris, D. (2000), Drowning in E-mail Overload?, available 
at: www.ferris.com/pub/FR-1 14-e-mailoload.html 
(accessed 19 February 2002). 

Financial Mail on Sunday (2000), “Communication tools 
are becoming more of a hindrance than a help”, 
Financial Mail on Sunday, 27 August. 

Flood, G. (2001), “Management: e-mail overkill is a 
corporate malady”, Computing, 22 February, p. 49. 

Goldschmidt, K., Folk, N., Callahan, M. and Shapiro, R. 
(2001), E-mail Overload in Congress: Managing a 
Communications Crisis, available at: 
www.congressonlineproject.org/e-mail.html 
(accessed 23 July 2002). i 

Hiltz, S.R. and Turoff, M. (1985), "Structuring computer- 
mediated communication systems to avoid 
information overload”, Communications of the 
ACM, Vol. 28 No. 7, pp. 680-9. 

Hogg, C. (2000), Internet and E-mail Use and Abuse, 
Short Run, Exeter. 

Kimble, C. and Abu Bakar, J. (2001), “Information 
overload and electronic mail: tagging message 
content — a partial solution?”, Proceedings of 6th 
UKAIS Conference, University of Portsmouth, 
Zeus Press, Manila, pp. 371-80. 

Kimble, C., Hildreth, P. and Grimshaw, D. (1998), “The 
role of contectual clues in the creation of 
information overload”, Proceedings of 3rd UKAIS 
Conference, University of Lincoln, McGraw-Hill, 
New York, NY, pp. 405-12. 

Lewis, D. (1999), Information Overload: Practical 
Strategies for Surviving in Today's Workplace, 
Penguin Books, Harmondsworth. 

Lyons, J. (2002), Working Parents “Spend Longer on 
E-mail", 19 May, Press Association, London. 

Mallon, R. and Oppenheim, C. (2002), “Style used in 
electronic mail", Aslib Proceedings, Vol. 54 No. 1, 
pp. 8-22. 

Mandell, M. (2001), “Preventing e-mail overload”, 
World Trade, Vol. 14 No. 3, pp. 44-6. 

New Shorter English Dictionary (1993), Oxford University 
Press, Oxford 

Nisse, J. (1997), “NU to make e-mail libel payout”, 

The Times, 18 July, p. 25. 

Oppenheim, C. (1997), “Managers’ use and handling of 
information”, International Journal of Information 
Management, Vol. 17 No. 4, pp. 239-48. 


179 


E-mail overload in the UK workplace 





Jenni Ingham 


Oxford English Dictionary Online (updated regularly), 
Oxford University Press, UK, available at: http:/ 
athens.oed.com/entrance.dtl (accessed 24 July 
2002). 

Pitney Bowes (2000), E-mail or E-stress?, available at: 
www. pitneybowes.co.uk/news/ 
article.asp?article=103 (accessed 24 January 
2002). 

Reuters (1996), Dying for Information: An Investigation 
into the Effects of information Overload Worldwide, 
available at: about.reuters.com/rbb/research/ 
dying.htm (accessed 24 January 2002). 

Reuters (1997), Glued to the Screen: An Investigation into 
Information Addiction Worldwide, available at: 
about.reuters.com/newsreleases/art_8-12- 
1997_id362.asp (accessed 13 August 2002). 

Sales Promotion (2001), “UK e-mail overload”, Sales 
Promotion, October, p. 4. 

Shedroff, N. (2000), “Forms of information anxiety”, 
Business 2.0, 28 November, p. 220. 


Aslib Proceedings 
Volume 55 - Number 3 - 2003 + 166-180 





Shenk, D. (1997), Data Smog: Surviving the Information 
Glut, Abacus, Tunbridge Wells. 

Sherwood, K.D. {n.d.), Tips for Overcoming E-mail 
Overload, available at: www.overcomee- 
mailoverload.com/advice/TopTentTips.html 
(accessed 24 July 2002). 

Silverman, D.A. (2001), Interpreting Qualitative Data: 
Methods for Analysing Talk, Text and Interaction, 
2nd ed., Sage Publications Ltd, London. 

Stewart, G. (2002), "Danger lurking in e-mail”, Business 
AM, 2 May. 

Whelan, J. (2000), E-mail@work: Get Moving with Digital 
Communication, Pearson Education Limited, 
Harlow. 

Whittaker, S. and Sidner, C. (1997), “E-mail overload: 
exploring personal information management of 
e-mail", in Kiesler, S. (Ed.), Culture of the Internet, 
Lawrence Erlbaum Associates, Mahwah, NJ, 
pp. 276-95. 

Wurman, R.S. (1989), Information Anxiety, Doubleday, 
Garden City, NY. 


180 





After the disaster: 
drawing up the 
insurance claim 





Robert Howes 


The author 


Robert Hawes is based at the British Library of Political 
and Economic Science, London School of Economics and 
Political SCence, London, UK. 

E-mail: R.Howes@lse.ac.uk 


<= #838383} 


University libraries, Disaster planning, Floods, Insurance, 
United Kingdom 


Ce 


Reports on the results of a major flood at the University of 
Sussex Library's repository at Lewes in October 2000, 
which severely damaged a collection of 42,000 books, 
14,000 bound periodical volumes, microfilms, 
newspapers, report literature and other materials. It 
describes the way in which the detailed insurance claim 
was drawn up and the methodology used to quantify the 
losses, usirig a combination of machine-readable data, 
card catalogues and estimation. It also describes the 
methodology used to calculate the cost af restoration and 
the value for replacement purposes. The outcome of the 
claim and future plans for collection development at 
Sussex are summarised. The conclusion draws some 
general lessons for future disaster prevention on how to 
minimise risks from flooding and gives advice on insuring 
library collections as well as drawing up insurance claims 
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Introduction 


On 12 October 2000, a flood of 
unprecedented severity submerged the 
University of Sussex Library book repository 
at Lewes in East Sussex, immersing the stock 
held there in river water, which was 
contaminated with mud, sewage and oil. The 
repository contained books, periodicals, 
newspapers, microfilms and unique special 
collections, which formed an important part 
of the library’s research collections. They had 
been outhoused at Lewes during prolonged 
building works for the extension and 
refurbishment of the main library and were all 
due to be returned to campus later in the year 
(Library Association Record, 2000). 

There is now a considerable body of 
literature on disaster planning, which includes 
a number of published reports on individual 
disasters and some advice on insurance. 
(Butler, 1986, 1987; Cady, 1999; Eastwood 
et al., 1978; Hammond, 1996; Kahn, 1994; 
Kennedy, 1995; Marrelli, 1983-84; Murray, 
1985; Naslund and George, 1986; Paradine, 
1995; Seal, 1984, 1985; Ungarelli, 1989, 
1990). With each such occurrence, lessons 
are learnt which may help to avoid or mitigate 
similar catastrophes in future. The purpose of 
this paper is to summarise the experience of 
the University of Sussex Library and what it 
has learnt from this disaster, particularly in 
the areas of disaster prevention and insurance. 


Disaster prevention 


The University of Sussex Library had a 
disaster plan which, while not specifically 
referring to the Lewes repository, covered the 
main questions which were expected to arise 
in an emergency. The plan contained 
emergency procedures, floor plans, lists of 
supplies and suppliers, priorities for salvage, 
plans for IT recovery, inventories of furniture 
and equipment, and excerpts from reference 
works on how to handle different types of 
library materials. The library had set up and 
trained a disaster team and had provided two 
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cupboards of supplies intended to deal with 
small accidents, such as burst pipes. 

In the event, the disaster was so 
catastrophic that most of this was of no use. A 
surge of water flowing down the River Ouse 
after a storm inland met the incoming high 
tide and burst over the river banks in a matter 
of hours. There were no staff in the repository 
at the time but eye-witnesses say that a small 
tidal wave rose up out of a nearby brook and 
completely flooded the repository within a 
couple of hours. Such was the pressure of 
water that the steel shutters on the loading 
bay were buckled inwards. Inside the results 
were spectacular. When staff were able to 
enter the repository on the fourth day after the 
flood they found a scene of complete 
devastation. The repository was a large single- 
storey warehouse, in which steel shelving had 
been arranged in two large blocks comprising 
12 rows, mostly of 12 double bays with seven 
shelves, reaching a height of 2.3 metres, with 
a smaller block of shelving near the loading 
bay. Most of the shelving was full of stock 
comprising 42,000 books, 14,000 bound 
periodical volumes, around 6,500 unbound 
periodical parts, 20 hardcopy newspaper 
titles, mostly in newspaper boxes but some in 
bundles, 16 newspaper titles on microfilm 
and various unique special collections, mostly 
in archive boxes or open pamphlet boxes. The 
force of the water had knocked over most of 
the shelving in the loading bay and in the first 
block towards the left, while in the rear block 
about two-thirds of the shelving had fallen 
over towards the right. Some of the stock, 
particularly the bound periodicals, had 
remained wedged on the shelves as it swelled 
up and expanded. The rest had fallen on the 
floor or was trapped by fallen stack. At the 
rear of the store, there was simply a huge pile 
of books about two metres high. The water 
had reached a depth of four metres and taken 
three days to subside, so everything had been 
thoroughly soaked. Many of the books had 
floated around, filling the aisles and ending up 
far from their original locations. Bound 
volumes of the eighteenth century House of 
Lords Journal were found in the entrance hall 
while other books were wedged in the pipes 
and heaters high on the walls. Amazingly, a 
bowl of paper clips on a table was completely 
-dry, so the table must have floated up on the 
rising tide and then been let gently down 
again as the water receded. 
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In these circumstances, there was no 
undamaged stock to rescue and nothing that 
the disaster team could have done by itself. 
Fortunately, as part of the disaster planning 
process, the library had registered as a priority 
user with both Harwell Drying & Restoration 
Services and Riley Dunn & Wilson. The 
Harwell representatives were on the scene on 
the day that staff first regained access to the 
building anc began removing one of the 
priority special collections the following day. 
Subsequently, at the request of the loss 
adjusters, Riley Dunn & Wilson arranged for 
the entire remaining stock to be removed to 
freezer storage, which took some three weeks 
in all to complete. By the end, many of the 
remaining volumes were covered in mould. 
The sheer amount of material involved meant 
that it was not possible to keep records of 
individual items. 

In retrospect, the main benefit of the prior 
disaster planning process turned out to be the 
knowledge acquired and the contacts made 
with salvage and restoration experts. Library 
staff had three days to consider what to do 
while they waited for the waters to fall and 
were to some extent psychologically prepared 
for the scale of damage they found when they 
were finally able to get back into the building. 
It was nevertheless an upsetting experience, 
particularly for the member of staff who had 
been responsible for the repository. 
Fortunately, because the repository was a 
remote closed-access store, day-to-day library 
services were largely unaffected (which was 
especially lucky as the flood occurred in the 
first week of the new academic session!). This 
reduced the immediate impact and allowed 
time for staff to plan the response to the 
disaster. 


Insurance 


The repository’s contents were insured under 
the university’s overall insurance policy with 
the Universities Mutual Association, UMAL, 
and reinsured with ACE Insurance S.A.-N.V. 
It soon became obvious that the damage 
would give rise to a large claim. For this 
reason, the loss adjusters changed their 
original advice that the stock should be 
jettisoned and insisted on removing it to 
freezer storage, even though mould had 
started to grow. A member of staff was 
charged with drawing up the claim and this 
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took up a large proportion of his time for the 

next six months. The claim was expected to 

run into millions of pounds and for such a 

large amount the insurers required as much 

detail as possible. There were two major 

aspects to this: 

(1) establishing exactly what was in the store; 
and 

(2) estimating how much it was worth. 


Like most university libraries, Sussex had a 
range of materials besides books and 
periodicals. The special collections comprised 
sheet music, technical reports, official 
publications, offprints, working papers and 
miscellaneous publications of all descriptions. 
Fortunately, the really valuable manuscript 
collections were housed safely in the strong 
rooms in the main library and were not 
affected by the flood. Most of the books in the 
repository formed part of the library’s main 
collection and had been out-housed during 
the building works on campus. They were all 
catalogued and it was a relatively 
straightforward task to produce an accurate 
number of books and a listing of the 
bibliographic details of each title. Books that 
were out on loan at the time and so escaped 
damage were deducted from the total. 

The library management system held 
catalogue records giving the bibliographical 
details of periodical titles, with holdings and 
locations. It also recorded individual bound 
periodical volumes but without any title 
information. It was thus possible to produce 
separately a list of the periodical titles and a 
count of the number of bound volumes 
damaged, but it was not possible to correlate 
the two. Furthermore, there were a 
considerable number of unbound periodical 
parts, which were not recorded on the 
computer because Sussex was at that time still 
using a manual check-in system. It was only 
possible to make an estimate of these based 
on the floorplan of the store, a sample count 
of shelves of unbound periodicals in the main 
library and some photographic evidence. The 
number of microfilms was calculated from the 
catalogue records and from accession cards. 
In the case of newspapers, we had records of 
the titles and dates held, so these were used as 
the basis of calculation. 

The special collections caused the greatest 
difficulty. Two relatively small book 
collections were fully catalogued, so accurate 
records could be obtained from the computer. 
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Another two collections were recorded on 
catalogue cards. Sample counts were done to 
produce an estimate of the total number of 
items in these collections. This was not 
entirely straightforward because many of the 
publications were serials or administrative 
reports and some catalogue cards had been 
used to record a number of issues. Two 
collections held on deposit had typescript 
catalogues, which provided fairly accurate 
numbers, and for another collection there was 
a typescript classification scheme, which gave 
an idea of the scope of the collection, 
although no numbers or bibliographic 
description. For the rest we had to rely on a 
small handbook to the special collections 
written eight years before the disaster, which 
gave estimated numbers for individual 
collections (Edgell and Ravilious, 1992). In 
addition, two architectural models which had 
associations with the original architect of the 
University of Sussex, Sir Basil Spence, had 
survived but in a damaged condition. 

There were two basic options for recovering 
from the disaster: restoration or replacement. 
With the exception of the newspapers and 
microfilms, which were so damaged that they 
had to be discarded, virtually all the contents 
of the repository were salvaged and put into 
freezer storage, where their condition was 
stabilised pending a decision on the insurance 
claim. The restoration contractors then 
provided estimates of the cost of cleaning, 
drying and restoring the damaged materials. 
In one case the estimated cost was based on 
the number of pages per volume (i.e. price 
ranges based on 100, 200, 300 pages, etc.). 
Since even our computer-held catalogue 
records do not give pagination, we were 
unable to give exact figures for each item and 
so had to calculate on the basis of an average 
of 200 pages for academic monographs, 400 
pages for bound periodical volumes and 100 
pages for periodical parts, literary works, 
official publications and report literature. The 
cost of restoration was then calculated using 
these average figures and the number of items 
as set out above. 

The cost of replacement was more difficult 
to calculate. In the end, it was decided to 
work on the basis of replacement using a 
mixture of new, second-hand and microfilm 
equivalents, depending on the particular type 
of material. For academic monographs, it was 
considered more appropriate to use the price 
of new books, since we were more likely to 
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want to replace lost works with the latest 
editions. Using a break-down based on 
classmark and the average figures for new 
academic books by subject produced half- 
yearly by the Library and Information 
Statistics Unit (LISU) at Loughborough 
University, we were able to produce an 
accurate estimate of the cost of replacing the 
main collection books. 

For periodicals, we sought replacement 
costs for individual titles and holdings (which, 
for the reasons explained above, we were not 
able directly to correlate with the cost per 
volume or part for restoration, although we 
were able to compare the relative total costs of 
both options). A list of periodical titles with 
holdings was output from the computer 
catalogue to a spreadsheet and then sent to 
the now-defunct Backruns Department of 
Dawson Books, who circulated it among 
specialist dealers worldwide. This took about 
three months, at the end of which Dawson’s 
returned the spreadsheet to us annotated with 
the offers for various titles that it had received. 
We obtained quotes for about a third of the 
damaged titles this way (although not always 
for identical runs) and from this were able to 
calculate an average price per title. A value for 
all the lost titles was then calculated on this 
basis. For the sake of comparison an average 
price was calculated for microfilm 
replacements, using the price list displayed 
online on the UMI Web site, and a total 
obtained by multiplying up. 

The newspapers were much more difficult. 
The Guide to Microforms in Print proved to be 
unreliable and it was necessary to check 
individual Web sites or contact individual 
suppliers by e-mail or fax. Some prices were 
obtained directly from suppliers’ Web sites, 
either for individual titles (e.g. UMI) or as a 
standard charge per roll, which was then 
multiplied by the number of rolls (e.g. Library 
of Congress), but in most cases a specific 
price quote was obtained from the relevant 
supplier. No microfilm equivalent for some 
titles could be identified and in these cases a 
standard figure of one roll per year was 
assumed and then costed using the Library of 
Congress standard charge. This resulted in 
what was probably a very conservative 
estimate. 

As described above, the special collections 
were very heterogeneous and so a variety of 
means had to be used to establish a 
replacement value. Most of the specialised 
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book collections were costed using the LISU 
figures for new books. A collection of Quaker 
books owned by the local Society of Friends 
was valued by a specialist bookseller. The 
value of a historic collection on weights, 
measures and trading standards was 
estimated by checking titles against the online 
union catalogue of second-hand bookdealers 
at abebooks.com. Some 7 per cent of books 
were found and an average price calculated by 
multiplying up and adding some extra for 
known valuable items. This method worked 
in this specialised area but would be difficult 
to apply more generally. When a sample of 
the main collection books was checked, prices 
for the same edition were found to vary by a 
factor of three and for different editions of the 
same work by factors of ten, making it 
difficult to establish a reliable average. It also 
seems likely that expensive works do not 
appear on this catalogue, skewing the price 
range in a downward direction. For official 
publications and pamphlets, which scarcely 
figure in the second-hand market, an average 
value was calculated based on the cost of new 
official publications currently purchased by 
the University of Sussex Library. For 
technical reports, the cost of microfiche 
copies from the British Library Document 
Supply Service was used. Individual quotes 
were obtained for other items, such as the 
damaged architectural models. 

Note that outsiders are often mystified by 
library jargon and librarians need to recognise 
that statistics kept for one purpose may not be 
of much use for another. For example, “item” 
provides an adequate term for measuring 
handling transactions, but for valuation 
purposes there is a great deal of difference 
between a bound periodical volume and an 
unbound periodical part or technical report. 
When talking about periodicals, it is easy to 
slip without thinking from the number of 
titles or subscriptions to the number of 
physical volumes or parts. Again, for 
valuation purposes, there is a great difference 
between the two but this is not immediately 
apparent to laypeople. It should also be 
remembered that in the UK, value added tax 
(VAT) is payable on most goods and services 
except for new or second-hand books and 
periodicals printed on paper. This affects the 
comparative costs of some options, for 
example, between restoration, which incurs 
VAT, and replacement by hardcopy, which 
does not, or between replacing periodicals 
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with second-hand runs (no VAT) and 
microfilm (VAT payable). As many 
microfilms are priced in dollars or euros, 
fluctuating exchange rates may also affect the 
sums claimed or paid. 

Besides the collections themselves, the 
flood caused other losses — notably the stack, 
which was distorted and contaminated and so 
could not be re-used, and various items of 
furniture and equipment. We had a detailed 
floorplan of the store at the time of the flood 
and so the number of shelving components 
could be accurately calculated. A price quote 
from a specialist contractor was obtained for 
the shelving, and trade catalogues used to 
price the other items of furniture and 
equipment. In retrospect, we might also have 
claimed for the house plants, which looked 
very sad after the flood! Whether a collection 
is replaced or restored, additional staff will be 
needed to process the stock and 
subscriptions. Staff costs were estimated on 
the basis of a two-year project requiring a 
project team composed of a professional 
librarian, a senior library assistant and two 
junior library assistants. These numbers 
represent one third of the collection 
management staff complement and recognise 
that it should be possible to reuse many 
bibliographic records. In point of fact, the 
length of such a project will depend on the 
recovery option chosen. If restoration is the 
preferred option, the rate of work will depend 
on the output of the restoration contractors 
and is likely to take longer than 
straightforward replacement but working on a 
part-time basis. Besides staff, equipment, 
such as computers, and other sundry 
expenses, such as binding and security tags, 
must be allowed for. 

The contractors who salvaged the 
collections from the repository sent in 
invoices, which were paid pending settlement 
of the claim with the agreement of the loss 
adjusters. In addition, there were substantial 
costs for the freezer storage of the damaged 
material and the final disposal of material that 
could not be restored. The rest of the 
university also had an interest in the 
collections, which have a potential to affect 
student recruitment, particularly of post- 
graduates, and the administration considered 
submitting a claim for financial loss under the 
“Business Interruption” policy, although in 
the event this was not pursued. 
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After a series of negotiations between the 
university, the loss adjusters, UMAL and the 
reinsurers, the claim was finally settled for 
£2.6 million. The settlement gave the 
university the freedom to restore or replace 
the damaged material, or to invest in new 
resources which would play an equivalent role 
in fulfilling its institutional mission. This 
recognises that research interests and teaching 
and learning methods have changed since the 
collections were first acquired. It also takes 
account of the practical difficulties of 
restoration: with the exception of some boxed 
items, the books in the store had become 
completely mixed up by the force of the flood 
and it was virtually impossible to identify 
individual items. It was therefore decided to 
restore a part of the Chester Music collection, 
which had survived relatively unscathed in 
boxes and had been salvaged early on before 
mould started to develop, but the remainder 
of the store’s contents had to be written off 
and disposed of to a landfill site. Although 
there were a number of items and collections 
which would have been worth restoring, the 
difficulty of locating them meant that the cost 
could not be justified. Individual items will be 
replaced where a case can be made and the 
rest of the money is being used to expand the 
library’s resources to meet the current needs 
of the university, particularly with enhanced 
electronic provision. Academics are being 
consulted for their views on replacements and 
new purchases. 


Lessons for the future 


The Lewes flood and its aftermath provide a 
number of lessons, which may be of interest 
to other libraries. 

The first lesson is the vital importance of 
having adequate insurance cover. It is 
advisable to check with the organisation’s 
insurance officer or agent to ensure that the 
costs of replacing or restoring stock, furniture 
and equipment are included, as well as the 
costs of processing and of maintaining 
business continuity. This is likely to involve 
making an estimate of the value of the 
collection, which may be particularly difficult 
in the case of unique archival collections. 
There may be a temptation to say that these 
are irreplaceable and therefore uninsurable > - 
but it is worth remembering that, with each 
major disaster, new lessons are learned about 
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restoration techniques and it is now possible 
to save many items, particularly water- 
damaged material, which previously would 
have been written-off. Organisations also 
need to decide if they are going to insure on < 
like-for-like basis and whether to cover the 
whole cost of their collections or only part. 

The second lesson is the value of having a 
written disaster plan. Although the plan had 
no direct effect on the outcome of the 
disaster, the reading and discussions that 
accompanied it were invaluable in helping us 
react to the flood. We already knew where tc: 
begin. As part of the planning process, we had 
registered as priority users with both Harwell 
Drying & Restoration Services and with Riley. 
Dunn & Wilson, which undoubtedly helped 
the salvage process. 

A third lesson is the importance of keeping 
accurate records and statistics. Full 
bibliographic records are obviously the 
preferred solution, but even estimates, 
provided that they are reasonably accurate, 
are better than nothing. Dust may have to be 
blown off old catalogue cards which have not 
seen the light of day for years. Insurers are not 
prepared to accept unsubstantiated claims, 
rough estimates, roundings-up or back-of-an- 
envelope calculations. All aspects of a claim, 
both numbers and valuations, have to have 
some basis in proven reality. We were lucky in 
that most of our records were held in the main 
library and so were not affected by the flood. 
but it is wise to keep copies of records in a 
separate building wherever possible in case 
the main building is damaged or destroyed. 
Records should also be kept of actions taken 
in the aftermath of a disaster. It may not be 
possible to keep track of every item, since 
speed is of the essence in salvaging wet stock, 
but taking photographs and videos provides 
evidence of the extent of the disaster and the 
measures taken to react to it. 

A fourth lesson is to be prepared to deal 
with professions and jargon with which most 
librarians do not normally come into contact. 
For example, there is an important difference 
between loss adjusters and loss assessors. Loss 
adjusters are usually appointed by insurance 
companies to deal with large claims. Their 
aim is to mitigate the loss and reach an agreed 
settlement by acting objectively between the 
two sides. They are paid by the insurance 
company and their costs do not come out of 
the claim. Loss assessors, on the other hand, 
act solely on behalf of the person or body 
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which has suffered the loss. They receive a 
percentage of the final settlement as payment 
and so their aim is to obtain the largest 
possible amount. Their costs are not 
recoverable, however, and the size of the final 
pay-out received by the claimant is 
accordingly reduced. It may be necessary to 
appoint a loss assessor if an agreed settlement 
cannot be reached but librarians should 
beware of rushing to sign up assessors who 
arrive on the scene in the immediate 
aftermath of a disaster. 

The final lesson concerns a simple but 
easily overlooked factor, namely the decision 
on location or the siting of the repository, 
which was the original cause of the disaster. 
The store was a modern rented warehouse 
located in a light industrial estate in what had 
once been the flood plain of the River Ouse. 
There had not been a flood in Lewes for 40 
years, but older inhabitants remembered how 
it used to flood. The university officers 
responsible for selecting the building were 
concerned with risks such as vandalism, burst 
pipes and excessive daylight, but not with the 
danger of external flooding, which was not 
immediately obvious. When selecting a site 
for a new library or library store, it is therefore 
important to look at maps, talk to local 
inhabitants and if possible go up to a high 
vantage point to look for potential dangers 
outside and away from the building. Whether 
in point of fact we would have acted 
differently is another matter, The building 
was so superior to the others on offer that a 
risk assessment of a five-year lease in an area 
which had not flooded for nearly half a 
century might well have led us to consider it a 
risk worth running. Nevertheless, as events 
subsequently proved, the danger of flooding 
should be taken into consideration when 
deciding on the location of a building. 
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Can remote Internet voting increase turnout? 
Susan Henry 


Keywords Elections, Government, Internet, 
Electronic data interchange, Local government, 
United Kingdom 


In an effort to increase turnout at elections the UK 
government has been piloting electronic voting. At 
the 2002 local elections five councils tested remote 
Internet voting for the first time. Swindon Borough 
Council conducted the largest pilot, offering 
remote Internet voting to all voters. Almost 15,000 
voters were surveyed as to their motivation for 
choosing their voting method. Turnout did 
increase by 3.5 per cent but it is impossible to state 
conclusively that this was due to the availability of 
the Internet voting option, since most Internet 
voters were already regular voters. Future pilots 
should allow a more conclusive assessment of the 
impact on turnout of remote Internet voting. 


The British Life and Internet Project: inaugural 
survey findings 

Barrie Gunter, Chris Russell, Richard Withey and 
David Nicholas 


Keywords Internet, Consumer behaviour, 
Interpersonal communications, United Kingdom 


The British Life and Internet Project was launched 
in 2002 to explore the way people use the Internet 
in Britain. It will survey people online throughout 
each year on a range of topics linked to the use of 
the Internet. In the first survey that was conducted, 
just before Christmas 2002, respondents were 
asked a series of questions about their history of 
involvement with the Internet and the nature of 
their online behaviour. The findings indicated that 
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online technology was used overwhelmingly for 
sending and receiving e-mails and for obtaining 
news and information linked to work and hobbies. 
Such is the demand of the Internet on the time of 
users that many acknowledged reducing the 
amount of time they spend on other activities, such 
as watching television, reading newspapers, and 
even going out to do the shopping. The Internet is, 
for most of its users, an important interpersonal 
communications medium, used to stay in touch 
and gossip with family and friends. 


On open source IR 
Andrew MacFarlane 
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Open source software development is becoming 
increasingly popular as a way of producing 
software, due to a number of factors. It is argued in 
this paper that these factors may have a significant 
impact on the future of information retrieval (IR) 
systems, and that it is desirable that these systems 
are made open to all. Some problems are outlined 
that may prevent the uptake of open source IR 
systems and a number of open source IR systems 
are described. 
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The appropriateness of utilising a training resource 
programme consisting of a mix of media and 
technologies for the training of cataloguing 
students is evaluated. The findings from reported 
research and evaluation of the training resource 
programme made it possible to identify advantages 
and disadvantages of using such a programme. The 
results of the research enabled the researcher to 
derive guidelines for the design and development of 
a training resource programme consisting of a mix 
of media and technologies. The use of media and 
technologies, in a training research programme for 
cataloguing training, can be utilised in training 
cataloguing students in contact classes, distance 
education as well as in-service training. 


The enterprise portal — is it knowledge 
management? 
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Currently we are in the middle of the information 
age, suffering from information overload on the 
one hand and a lack of knowledge on the other. 
Enterprise portals (EPs) are seen as the antidote to 
these problems by becoming more and more the 
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ultimate knowledge management (KM) tool. The 
current hype about EPs is focused on their 
application as KM tools. Very little attention is 
given to other aspects of KM, namely the 
organisational, human and cultural aspects. The 
article will provide an overview of the technical and 
strategic relationship between EPs and KM and 
illustrate that EPs are only the technology 
component and should not be mistaken for the 
essence of KM. What is needed for successful KM 
in an organisation is not technology alone, but also 
a knowledge-sharing culture, knowledge-sharing 
policies, organisational processes, performance 
measurement and business strategies. 
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Seeding a community of interest: the experience 
of the knowledge library project 


Andrew Cox, Keith Patrick and Rahman Abdullah 
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The paper reports on the early lessons learned in ` 
the process of forming an online community for 
librarians from different organisations and sectors 
to discuss knowledge management. Time, critical 
mass, diversity of membership, focus, information 
and communication technology, incentives, 
concepts of membership, services and leadership 
were all factors in the slow coalescing of the group 
as an active self-sustaining network. 
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| Abstract | 

In an effort to increase turnout at elections the UK 
government has been piloting electronic voting. At the 
2002 local elections five councils tested remote Internet 
voting for the first time. Swindon Borough Council 
conducted the largest pilot, offering remote Internet 
voting to all voters. Almost 15,000 voters were surveyed 
as to their motivation for choosing their voting method. 
Turnout did increase by 3.5 per cent but it is impossible to 
state conclusively that this was due to the availability of 
the Internet voting option, since most Internet voters 
were already regular voters. Future pilots should allow a 
more conclusive assessment of the impact on turnout of 
remote Internet voting. 
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Background 


In the USA the development of electronic 
voting methods, including Internet voting, 
has been driven by a desire to find alternatives 
to the out-moded manual and mechanical 
systems currently used. In the UK, where 
voting has always been by paper ballot, a 
system in which the electorate has confidence, 
the impetus for the move towards electronic 
voting is primarily a desire to increase 
turnout. 

A high turnout in an election is an 
indication that the electorate is engaged with 
the political process and gives elected officials 
a stronger mandate in office. Turnout in the 
UK, at both general and local elections, has 
been declining in recent years. In the 2001 
general election turnout was only 59 per cent, 
a fall of 12 per cent from the previous general 
election in 1997; it was the lowest turnout 
since 1918. While most non-voters, when 
asked, will cite a lack of knowledge or interest 
in the political process, or express the opinion 
that their vote will have little impact on the 
outcome, some also cite the inconvenience of 
the voting procedure (Diplock, 2001). The 
current government, as well as exploring ways 
in which to engage potential voters in the 
political process, has also been keen to 
consider new methods which might 
encourage a higher turnout by making it 
easier and more convenient to cast a vote. 

In the 2000 local elections 32 local 
authorities piloted variations of electronic 
voting and counting and all-postal voting. 
The electronic voting methods were not very 
successful in increasing turnout since they still 
required voters to attend a particular place on 
a particular day, though the counting process 
was speeded up considerably. The all-postal 
voting pilots were very successful in terms of 
an increased turnout. In Swindon, the four 
wards that tested all-postal voting saw a 
dramatic increase of turnout. Penfield ward, 
for example, had a turnout of 19.5 per cent in 
1999, which increased to 30.9 per cent in 
2000. 

The success of these pilots, in terms of 
achieving a secure and efficient election, led 
the government to pilot other methods and 
refine the already piloted methods. The 





Received 16 February 2003 
Revised 16 April 2003 
Accepted 21 April 2003 


193 


Can remote Internet voting increase turnout? 


Aslib Proceedings 





Susan Henry 


primary aim of the 2002 pilots was to find 
methods that were: 


... readily accessible to all electors given today’s 
lifestyles and the work and family commitments 
so many people have (DTLR, 2001). 


The Department of Transport and Local 
Regions (DTLR)[1], who oversaw the pilots, 
considered that remote methods such as 
telephone or Internet voting might be the best 
way of achieving the government’s aim of 
making voting as convenient and as attractive 
as possible. Early voting (i.e. for a period prior 
to the national polling day) was also tested for 
this reason. In the 2002 local elections early 
remote Internet voting was piloted by five 
authorities, early telephone and text message 
voting, as well as variations of electronic and 
all-postal voting, were also tested by a total of 
30 authorities. 

Opinion about Internet voting as an 
alternative voting method is divided. Remote 
Internet voting, casting a vote via any 
computer located somewhere other than in a 
designated polling place, is particularly 
contentious because there is no possibility of 
checking who is casting the vote, whether they 
are doing so under duress or whether the 
computer or voting software has been 
subjected to any tampering or interference. In 
the USA, where Internet-enabled elections 
have taken place, organisations, such as the 
Voting Integrity Project[2], have opposed 
them believing that as long as Internet voting 
is not the only alternative method available 
and while it does not offer security and 
privacy for voters it should not be used 
(Phillips and Jefferson, 2000). The Voting 
Integrity Project favours methods such as 
postal voting until such time as the digital 
divide has been bridged or is not so wide and 
is particularly wary of remote Internet voting, 
which it believes is very vulnerable to 
breaches of security and privacy. 

Companies, such as Election.com Inc., who 
organised the first Internet-enabled election 
(the Arizona 2000 Democratic Primary) 
counter that the problem of a digital divide 
can be, and in Arizona was, overcome 
through an “aggressive outreach program” 
and by providing a range of ways for the 
electorate to cast its vote (Mohen and 
Glidden, 2001). 

A Californian task force, headed by Bill 
Jones, investigating the feasibility of Internet 
voting, came to a similar conclusion as that of 
the Voting Integrity Project (California 
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Internet Voting Taskforce, 2000). It found 

that the barriers, including legal and fiscal 

issues as well as security and practicality, 

make remote Internet voting unfeasible at 

present. It suggests that Internet voting 

should be phased-in over four stages. The 

report does not offer a time scale but 

recommends that, as each stage is successfully 

attained, the next should be tested. The four 

stages suggested are: 

(1) Internet voting at polling stations; 

(2) Internet voting at any polling station; 

(3) remote Internet voting from county 
computers or kiosks; and 

(4) remote Internet voting from any Internet 
connection. 


The first Internet-enabled election in the UK 
was effectively starting at the fourth stage, 
potentially the most high risk Internet voting 
method, though certainly the one offering 
voters the most convenience, particularly 
when combined with an extended voting 
period. 


Aims, objectives and scope 


This study intended to: 

+ Examine the feasibility of Internet voting. 
This method had not been used in a 
public election in the UK before and so 
was new to voters. Does it work? Does it 
offer voters a secure, secret and 
accountable voting method? 

+ Ascertain whether it is attractive to voters. 
Were voters willing to use the method? 
Was it easily understandable? Did they 
have confidence in it? 

+ Determine whether it will increase turnout by 
attracting new voters. Did more people 
vote than in past years and were a 
significant number of votes cast online by 
previous non-voters? 


Focus on Swindon 

The research focused on Swindon because it 
offered the largest coverage (126,953 voters) 
of all the pilot schemes. Swindon was 
piloting Internet voting in all 19 of its wards 
that were holding an election, whereas the 
other councils were only piloting in two or 
three wards. Swindon voters had the choice 
to vote via the Internet, the telephone, by 
post or in the traditional manner at the 
polling station. 
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Swindon is situated in the South West of 
England, and is about an hour from London 
by rail. It has good rail and road links; the M4 
passes close by, making it very accessible and 
therefore attractive both for employers and 
employees. The railways used to be the major 
employer but now large manufacturing 
companies and several multi-national 
companies have premises there. The 
population is increasing, attracted by the low 
unemployment rate and the new housing 
developments. The current electorate stands 
at 141,793 and is increasing by about 2,000 
voters a vear. The borough includes Swindon 
itself and several small nearby villages. 


Swindon Borough Council pilot 

The council was offered a choice of voting 
suppliers and chose Electoral Reform 
Services. This company is part of the 
Electoral Reform Society, which has been 
involved with UK elections since 1884. It has 
its own telephone voting system, which was 
used in Swindon. The Internet voting element 
of the pilot was contracted out to a US 
company, VoteHere Inc. This company has 
been involved in many US online elections, 
both public and private. 

Those eligible to vote (126,953 people) 
received a ballot pack including a sealed 
polling card. The pack contained details of 
the various methods of voting and the sealed 
polling card included a unique ballot code 
consisting of two sets of numbers, one of six 
and one of four digits, randomly generated. 
This ballot code could be used to vote either 
via the Internet or by telephone during the six 
day early voting period (6a.m. 25 April to 
12a.m. 30 April). If the voters did not vote via 
either of these methods they could take their 
ballot card to the polling station and vote in 
the traditional manner on the national polling 
day on 2 May. Those who had applied for a 
postal vote had their ballot code number 
deactivated prior to the opening of Internet 
and telephone voting. 


The Internet voting process at Swindon 

(1) Voters log on to the voting Web site by 
entering their unique ballot code. 

(2) Screen two. They are prompted to enter 
their date of birth. This does not identify 
individual users and has no effect on the 
process other than to confirm that voters 
are aged over 18. It is intended to 
reassure voters with another level of 
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security and has been found in the past to 

deter fraudulent voting attempts. 

Screen three. A facsimile of the ballot 

paper is then displayed. Voters make their 

selection by clicking in the appropriate 
box. 

Screen four. The details of their date of 

birth, ward they are voting in and the 

candidate they are voting for appear on 

screen. At this stage it is possible to go 

back to screen three and change the vote 
if the voter so wishes. If they are satisfied 
with the details of their vote they click the 
submit button. After clicking the submit 
button voters cannot alter or retract their 

Internet vote. 

(5) The vote is encrypted and saved. 

(6) Screen five. The final screen displays the 
time and date of voting (though not the 
candidate voted for) and the voter can 
print it out if they wish. The voter clicks 
the log out button and voting is 
completed. 

(7) In the case of the Swindon election, 
voters were forwarded to a questionnaire 
after logging out. 
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At the close of early voting, the ballot codes of 
all the telephone and Internet voters were 
received by the council from Electoral Reform 
Services and the names were marked on the 
electoral register in order that those voters 
could not also vote at the polling station. 


Methods 


The methods used in the research were 
questionnaires, a psephology study of voter 
turnout and an interview with the pilot 
co-coordinator at Swindon, Mr Alan 
Winchcombe. 


The questionnaires 

The electronic voting and all postal pilots in 
2000 were partly assessed by an exit poll 
questionnaire and this study used the same 
method, since it can provide good 
quantitative data with which to assess the 
issue. Obviously, an exit poll in an 
Internet-enabled election presents problems 
in that it is hard to approach the voters if they 
are voting remotely. Therefore three online 
questionnaires were devised, one pre- and the 
other two post-election. A printed polling 
station questionnaire was also compiled to 


195 


Can remote Internet voting increase turnout? 


Aslib Proceedings 





Susan Henry 


gauge opinion among those who used the 
traditional voting method. The second, third 
and fourth questionnaires were intended to be 
completed only by voters and the first 
(pre-election) questionnaire was available to 
and promoted as a forum for voters and 
non-voters alike. 

The opinion of non-voters is important in 
such a study but they are hard to identify and 
canvass. The restrictions of the research 
unfortunately did not allow for the collection 
of non-voter opinion besides that captured in 
the first questionnaire. 

The questionnaires were kept as brief, 
though as thorough, as possible, so that 
people would not be put off responding by 
being asked to complete a long questionnaire. 
In order to achieve as high a response rate as 
possible most questions included tickable 
options but also allowed space for 
respondents to write in their own answer if 
appropriate. 

The four questionnaires shared many 
questions in common. These were intended 
to establish who was voting (age, gender and 
occupation), how often they vote (local and 
general elections) and, in the case of 
questionnaires one and two, where and how 
often they use the Internet. 


Questionnaire one — pre-election survey 

The first questionnaire was an online one, it 

was available from 12 April to 2 May 2002, 

polling day, and was promoted in the 

Swindon local press (hard copy and online) 

and by Swindon Web sites. Its purpose was: 

+ ‘To gauge opinion about Internet voting 
before the method had been tried. 

* To see where people were getting their 
information about Internet voting. Was it 
online or in hardcopy, was it coming from 
the council, media sources or from 
political parties? 

* To allow a comparison of impressions 
about security and secrecy between those 
who had not yet experienced Internet 
voting and those who had. Obviously it 
cannot be a direct comparison since the 
answers: are anonymous and it is not 
possible to be sure that respondents are 
the same people. 


Questionnaire two — Internet voters 

The second questionnaire was also an online 
one, available during the Internet voting 
period, i.e. from 25-30 April. It was accessible 
only from the VoteHere Web site. After 
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casting their vote Internet voters were 

automatically forwarded to the questionnaire 

Web site and could choose whether to 

complete the questionnaire or not. This 

questionnaire was not promoted or advertised 

anywhere else since it was purely for those 

who had voted via the Internet. Its purpose 

was to see: 

°- Who chose to vote via the Internet (age, 
gender and occupation), and whether 
they were regular Internet users and 
voters. 

¢ How easy a process they found it. 

* Whether they were confident in it as a 
method of casting a vote. 

¢ Whether they would use it in future 
elections. 


Questionnaire three — polling station voters 

The third questionnaire was a paper one, 

distributed to all voters at each polling station 

on Election Day, along with a prepaid return 

envelope. Its-purpose was to determine: 

+ Whether these voters were aware of the 
pilot methods of voting. 

* If so, why they chose to vote in the 
traditional manner. 

¢ Whether they agreed with the need for 
supplementary voting methods. 


Questionnaire four — Internet voters follow-up 

This was the follow-up questionnaire for 

Internet voters who had voluntarily left their 

e-mail address at the end of questionnaire 

two. It was devised in the light of those 

answers and was intended to clarify some of 

them, in order to: 

¢ Determine whether the option of early 
voting or of Internet voting was the 
primary reason for choosing Internet 
voting. 

+ Find out where people voted from: home, 
work or elsewhere. l 

+ Find out if they would be happy to vote 
from work if given the opportunity. 


Psephology study 

The psephology study was an examination of 
voting statistics for Swindon in the 2002 and 
previous elections to see if turnout had 
increased. It was also an examination of the 
political and socio-economic factors that may 
have influenced turnout. 


Interview 
Interviews with the co-ordinators of the pilots 
were a valuable element in the reports of the 
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2000 pilots, giving the perspective of the 
organisers on the trials which, along with the 
voters’ views, helped to clarify the feasibility 
of online voting. After the election an 
interview with Mr Alan Winchcombe, deputy 
returning officer and pilot coordinator, was 
conducted in Swindon. The interview was an 
informal interview/discussion over the course 
of five hours and a wide range of topics in 
relation to the pilot was discussed. It provided 
insight into the local political situation and to 
the socio-economic make-up of the various 
wards. 


Results 


Questionnaire one — pre-election survey 
This had a low response rate; there were only 
104 responses. This may have been due to the 
questionnaire not being promoted on the 
council Web site, which was being upgraded 
at the time. The majority of respondents were 
male (64.4 per cent) and in social classes A, B 
and C1 (79.7 per cent). They are frequent 
users of the Internet at home and at work, 
52.8 per cent vote at every local election, 33.7 
per cent at most local elections and 80.9 per 
cent vote at every general election. 

The majority (67.3 per cent) did intend to 
vote via the Internet, and of these 72.9 per 
cent were male and 55.7 per cent were in the 
25-44 age group. Most of the information 
about Internet voting was received in hard 
copy (70.5 per cent), 48.7 per cent came from 
the council, while political parties accounted 
for 7 per cent. The chief source of online 
information was also the council at 14.8 per 
cent. The media accounted for a total of 18.3 
per cent of information received, 3.5 per cent 
came from online sources. 

The respondents were more confident than 
not that their Internet vote would remain 
secret once cast (72.2 per cent), 8.7 per cent 
expressed a lack of confidence. Confidence 
that there would be no misuse or alteration of 
the vote once cast was higher, 76.9 per cent 
expressed confidence and 2.9 per cent were 
not confident. Respondents supported the 
need for supplementary voting methods (88.5 
per cent) such as Internet and telephone 
voting. 


Election results 
The turnout in the election was 31.23 per 
cent: 39,650 of the 126,953 people eligible to 
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vote did so. This was an increase of 3.5 per 
cent from the turnout in 2000 of 27.7 per 
cent. Table I shows the breakdown of the 
percentage of votes cast by each method. 

The figures in Table I show that Internet 
voting had a good take-up rate among the 
electorate as a whole and was twice as popular 
as telephone voting. Internet take-up in the 
individual wards varied considerably and will 
be examined later. 


The Internet voters 

Response to questionnaire two, the Internet 
voters’ “exit poll”, was good: 3,310 voters (77 
per cent) completed it. The government had 
hoped to attract young people with the new 
voting methods; however, they only 
accounted for 6.2 per cent of Internet voters 
and were outnumbered by those aged 60+. 
Take-up of telephone voting among the 
young was also disappointing. 

The results show that the majority of 
Internet voters were regular voters, 82.3 per 
cent vote at all or most local elections, Of the 
remaining 17.7 per cent, 4.7 per cent were 
first-time voters. The age distribution of first- 
time voters (Table II) indicates that most 
(64.1 per cent) are too old to be first-time 
voters due to age. Other explanations might 
be that some are foreign nationals voting for 
the first time or that these voters were 
motivated to vote for the first time because of 
the Internet voting option. However, it is 
impossible to be certain of the reasoning. 

When asked if the option to vote via the 
Internet had influenced their decision to vote 
in the 2002 election, 24 per cent said it had 


Table | Vote share of each voting method 





Voting method Votes cast % of total 
Telephone 2,028 5.11 
internet 4,293 10.83 
Postal 4,416 11.14 
Polling station 28,913 72.92 
Total 39,650 100 





Table Il Age distribution of first time voters (Internet 
only) 


Age Count % 


18-24 56 35.9 
25-44 76 48.7 
45-59 20 12.8 
60+ 4 2.6 


Total 156 100 
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influenced their decision while the remaining 
76 per cent said they had been somewhat 
influenced. This contradicts the regularity of 
voting figures since the regular voters (82.3 
per cent) would presumably have voted 
anyway and so would be expected to have 
answered no to this questicn. 

The majority of Internet voters found the 
process straightforward — 94.5 per cent rated 
the system as very easy to use. However, 
several voters commented that their preferred 
browser was not compatible with the system, 
a problem overcome by rebooting and using a 
Microsoft browser. Apple Mac users had a 
similar problem. One blind woman found the 
site hard to navigate using just her keyboard. 
The majority of comments were positive 
about the system. Only 0.2 per cent of 
Internet voters found the process less 
convenient than going to the polling station. 
Asked to specify what made it more 
convenient 1,412 respondents wrote in 
reasons (shown in Table IID. 

The primary reason cited for the increased 
convenience afforded by Internet voting 
among all Internet voters was that the poll 
was open over six days. However, among the 
age groups reasons for the convenience of 
Internet voting differed. For those aged 25-44 
and 45-59 it was being able to choose when to 
vote over a six-day period, but for the 18-24 
year olds it was being able to vote although 
away from home (many mentioned being 
away at university). While among those aged 
60 and over it was being able to vote without 
having to leave home/work. Avoidance of the 
polling station, a category which included 
having to make a special journey there, the 
difficulty of parking and not wishing to be 
pestered by pollsters outside, was in the top 
three reasons among each age group. 


Table Ill Reasons for Internet voting convenience 


Reason Count 
Poll open over six days 272 
No leaving home/work 244 
Avoid polling station 230 
Timesaver 155 
Work long hours 134 
Away from home 124 
Dependants at home 77 
Using Internet anyway 70 
Disabled/ill 45 
Effortless/simple 40 
Novelty 21 
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Confidence that the vote would remain 
secret and would not be misused or altered 
once cast was high; only 4.9 per cent 
expressed a lack of confidence about secrecy 
and 3 per cent were not confident that there 
would be no misuse or alteration of cast votes. 
Voters could have stopped the process at any 
time and voted in the traditional manner, so 
even though a few were not confident about 
the process they still chose to use it. 

The Internet voters were asked whether 
they would be happy to use the method in 
future elections, more were prepared to use it 
in future local elections (97.2 per cent) than 
in general elections (92.9 per cent). Those 
who were undecided or would not use 
Internet voting in a general election had two 
main reasons: security concerns; and a lack of 
occasion afforded by voting remotely. The 
quotes below may point to some of the 
explanations for the different attitude: 


Whilst I am confident of no tampering for local 
elections, who’s to say that it wouldn’t PAY to 
do so for general elections? (male 25-44). 


E-voting lacks the sense of occasion that 
sometimes accompanies a general election. 
(female 60+). 


Internet voters follow-up questionnaire 
The follow-up questionnaire was used to 
clarify a few issues about Internet voting; such 
as whether voters had chosen to use it because 
it was possible to vote over six days or because 
of the method itself, where they had voted 
from and whether they would be happy to do 
so from work. Of the 985 voters who left their 
e-mail address, 100 addresses proved to be 
invalid and 470 voters replied to the online 
questionnaire, a response rate of 53.1 per 
cent. Most Internet voters (50.5 per cent) 
cited the opportunity to vote via the Internet 
as their primary reason for using the method, 


% of total (n= 3,310) Valid % (n= 1,412) 





2.39 19.26 
2.14 17.28 
2.02 16.29 
1.36 10.98 
1.18 9.49 
1.09 8.78 
0.68 5.45 
0.61 4.96 
0.39 3.19 
0.35 2.83 
0.18 1.49 


198 


4 


Can remote Internet voting increase turnout? 


Aslib Proceedings 





Susan Henry 


only 14.3 per cent chose it because it allowed 
them to vote early, the remaining 35.2 per 
cent said both were of equal importance. So, 
although being able to vote over six days 
made the process more convenient for the 
majority of voters it was not their primary 
motivation for using this method to vote. 

Most voters voted from home (81.7 per 
cent) and did so because it offered greater 
convenience. This supports the criticism that 
remote Internet voting disadvantages the 55 
per cent of the population without access to 
the Internet and favours those in classes A, B 
and C1. Almost 50 per cent of voters would 
be happy to vote from work and so, if 
employers were to provide the facilities or 
allow the employees to use their work PCs for 
voting, it might address the imbalance. 

However, 50.2 per cent are not happy to 
vote from work or do not have a workplace, 
and privacy is an issue for almost half of this 
group. Greater promotion of the use of 
Internet facilities in libraries or Internet 
cafes/kiosks for voting might be considered, 
although this would still disadvantage 
cafe/kiosk users somewhat because they 
would have to pay for access. Also, there is a 
privacy issue since most Internet facilities in 
libraries and cafes are open plan and the 
choice of a voter’s candidate might be visible 
to others. 


The voter profiles 

The profile of the polling station voter was 
markedly different to that of the Internet voter 
(Table IV). Polling station voters are older 
and more are female, they also vote more 
frequently, though both groups have a high 
participation rate. The Internet voters are 
younger and more likely to be male; most are 
in social classes A, B and Cl. 


The polling station voters 

The questionnaire had a 39.44 per cent 
response rate, 11,404 of the 28,913 polling 
Table IV Voter profiles 


Characteristic 


Internet voters 
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station voters completed it. The results show 
that while the voters were aware of the option 
to vote via the Internet they did not choose to 
use it because, among other reasons, they just 
prefer to use the polling station (47 per cent), 
have no Internet access (29.2 per cent), or 
just find it more convenient to use a polling 
station (20.2 per cent). 

Attitudes towards Internet voting were 
demarcated along age lines. Among all age 
groups those aged 60+ composed the largest 
group who preferred to vote at the polling 
station (50.8 per cent), found the polling 
station more convenient (40.55 per cent) and 
had the lowest rate of Internet access (62.93 
per cent). They were also the group most 
daunted by the technology (58.10 per cent), 
although only 2.5 per cent of all respondents 
ticked this reason for not voting via the 
Internet. Some polling station voters, 9.1 per 
cent of all respondents, said that they had 
intended to vote via the Internet but had 
variously forgotten to vote via the Internet, 
had missed or were unaware of the deadline. 
Of this group almost half (49.31 per cent) 
were aged 25-44 and 36 per cent were aged 
45-59. 

The majority of polling station voters (60.7 
per cent) did agree that there was a need for 
supplementary voting methods, however the 
60+ age group was least in favour, 46 per 
cent of that age group agreed and 39 per cent 
disagreed. Comments in favour of 
supplementary voting methods such as 
Internet and telephone voting included that 
they would be helpful for the disabled and ill, 
also that they might encourage more people 
to vote. Those undecided or against 
supplementary methods disagreed and were 
of the opinion that apathy is the cause of a 
low turnout and new voting methods cannot 
address this problem. Security concerns 
were also an issue among those disagreeing 
with the need for supplementary voting 
methods. 


Polling station voters 








(%) (%) 
Largest single age group 25-44 48.0 60+ 42.6 
Social classes A, B and C1 69.2 52.2 
Female/male 43.3/56.7 52.3/47.7 
Vote at all or most local elections 82.3 ; 94.6 
Vote at all or most general elections 91.5 98.0 
First time voter 4.7 1.4 
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A secure voting method? 

There were no reported incidents of security 
breaches during the period of Internet voting 
although VoteHere personnel were 
monitoring the site and did observe several 
unsuccessful attempts to hack into it. 
Multiple votes from single IP addresses were 
also checked and found to be legitimate votes 
cast over Swindon companies’ systems. The 
council had encouraged local businesses to 
provide facilities for Internet voting in the 
workplace. 

There was a case at a polling station of a 
woman unable to cast a vote because her 
husband had inadvertently used her ballot 
code to cast his vote via the Internet. She was 
marked on the electoral register as having 
already voted by other means and so was 
refused a ballot paper at the polling station. 
Her husband, who was accompanying her, 
was also refused a ballot paper since he 
admitted to the polling staff that he had 
already voted in the election, albeit using his 
wife’s vote. This mistake was possible because 
the date of birth validation, requested during 
the logging on process, did not require the 
voter to enter their actual date of birth, only 
one that confirmed them to be aged 18 or 
over. 


Internet take-up among the wards 
Turnout in the Swindon wards has always 
varied in each election, and in the 2002 
election it varied between 21.78 per cent 
(Parks) and 52.68 per cent (Old Town and 
Lawn). The borough average was 31.2 per 
cent, an increase of 3.5 per cent. All wards 
had an increase in turnout in 2002 except 
those that had piloted all-postal voting in 
2000. The turnout in these wards had 
increased considerably with all-postal voting 
and it is perhaps unsurprising that it was not 
sustained in this election. The average 
Internet take-up in Swindon was 10.83 per 
cent. The wards with the highest Internet 
take-up, such as Abbey Meads and Shaw and 
Nine Elms (both almost twice the borough 
average), are more affluent, owner occupied 
areas with a younger population. As the 
MORI figures show, Internet access is likely 
to be high in such areas (MORI, 2002). 
Usage is 36 per cent (A, B) and 35 per cent 
(C1) by class and younger people have greater 
access; those aged between 25 and 54 account 
for 67 per cent of all Internet users. 
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Conversely, Parks and Penhill are socially 
deprived and had both a low turnout and a 
low Internet take-up rate. Parks was in fact 
the lowest in both categories with a turnout 
of 21.78 per cent and an Internet take-up 
rate of 5.74 per cent. Access and usage 
amongst C2 (18 per cent) and D, E (12 per 
cent) is the lowest of all social classes. The 
ward with the highest turnout was Old Town 
and Lawn. This population is affluent but is 
mainly composed of retired people. This 
group tends to vote regularly but generally 
has low rates of Internet use and access (4 
per cent among those aged 65 or more). 
Internet voting only accounted for 10.96 per 
cent of votes cast in this ward, similar to the 
borough average. 


A political explanation for the increased 
turnout? 

The political situation in Swindon may 
account for the increase in turnout. Swindon 
Borough Council had been Labour controlled 
from 1973 to 2000 when the party lost control 
and no other party won overall control. 
During Labour’s 27-year tenure turnout, as in 
the country as a whole, steadily declined, 
Labour councillors held most of the seats and 
held most with a high majority. As a 
Department of Employment, Transport and 
the Regions study found, turnout is generally 
lower when one party is in control of.the 
council and turnout declines further when 
that party has a large majority of the vote 
(DETR, 2000). However, when control of the 
council is “up for grabs”, as in Swindon in 
2002, party workers put greater effort into 
mobilising their supporters out to vote and an 
increase in turnout may result. 


Conclusions 


Is remote Internet voting feasible? 

The Internet pilot worked well overall. There 
were some accessibility issues relating to 
browser incompatibility and, more seriously, 
a visually impaired woman found navigating 
the voting Web site via her keyboard was 
difficult. The system is not foolproof as the 
voter who used his wife’s vote found. 
Nevertheless, most Internet voters found the 
system easy to use and were confident that 
their vote would remain secret and would not 
be misused or altered once cast. 
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The Electoral Commission states in its 
executive summary report of the pilots 
(Electoral Commission, 2002) that: 


The benchmark against which innovative pilot 
schemes should be tested is not a 100 per cent 
secure system. There are security and other 
weaknesses inherent in traditional polling station 
voting. The key issues here relate to voter 
confidence. The traditional system of voting by 
pencil and paper at polling stations has a very 
high degree of voter confidence, despite its 
inherent flaws. 


By this measure, Internet voting is currently 
feasible, confidence that it offered a secure 
way to cast a vote was high amongst users. 
However, that confidence was based upon 
ignorance of the true nature of the security 
features. A more secure means of validating 
the legitimacy of remote Internet voters needs 
to be developed before it is used on a wider 
scale. However, if infrequent voters are not to 
be discouraged from using the method, any 
means of voter validation has to be as simple 
and convenient as possible, which rules out 
pre-registration, for example. 


Is Internet voting attractive to voters? 
Those who cast their vote via the Internet 
were satisfied with the process. They found it 
easy to use and considered that it offered 
great convenience, most voted from home. 
They also expressed a willingness to use it in 
future lacal elections, though a slightly 
smaller number would use it in general 
elections. Those who had reservations about 
using it for general elections mostly said that 
they would prefer to vote at a polling station 
because general elections have a sense of 
occasion lacking in local elections. Some 
respondents suggested that Internet-enabled 
general elections might be a more attractive 
target for attempts at fraud or disruption than 
would local elections. Internet voting is not 
attractive to all voters and there is evidence of 
a digital divide. Those who chose the polling 
station did so because they prefer it or 
because they have no Internet access, while 
one-fifth claimed using polling stations was 
more convenient. The government’s intention 
to attract the under 25s out to vote with ` 
Internet voting was not realised. They also 
failed to use the telephone voting option in 
any great numbers. 

It can be concluded that Internet voting is 
predominantly attractive to a certain type of 
voter: affluent younger males who regularly 
vote and who use the Internet frequently. 
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However, as Gibson et al. (2002) observe, 
newer Internet users are increasingly less 
likely to be male and in classes A and B 
(Gibson, 2002). It is to be hoped that as 
Internet take-up increases, future Internet 
voting will appeal to a broader range of 
users. 


Did Internet voting increase turnout by 
attracting new voters? 

The turnout in 2002 did increase but it is 
inconclusive as to whether the increase of 3.5 
per cent was due to the Internet option being 
available. All wards, except those that had 
piloted all-postal voting in 2000, experienced 
an increase in turnout and in some wards it 
was a substantial increase. Only one ward, 
Shaw and Nine Elms, had an increase in 
turnout (2.4 per cent) of less than the 
borough average; it also had the second 
highest Internet take-up of any ward. 

The results, however, indicate that most of 
those choosing to vote via the Internet were 
regular voters already. Only 17.7 per cent 
were infrequent or first-time voters. It is 
impossible to determine how many of the 
first-time voters (4.7 per cent) were 
encouraged to vote for the first time solely 
because of the Internet option. The 
percentage of votes cast online was 10.83 per 
cent borough wide, though it varied in the 
wards from a high of 20.84 per cent to a low 
of 5.74 per cent. The variance illustrates that, 
as discussed above, Internet voting is 
attractive to only certain sections of the 
electorate. Telephone voting was only half as 
popular as Internet voting; only 5.11 per cent 
of voters used it. The variance among wards 
of telephone use supports the theory that 
Internet voting is only attractive to a certain 
section of the population since there was a 
much smaller variance in take-up of telephone 
voting (by ward the highest take-up was 6.08 
per cent and the lowest was 4.03 per cent), 
which indicates a more uniform take-up, 
albeit in smaller numbers. 

It is impossible to say with any certainty 
that the option to vote via the Internet 
resulted in the increased turnout in Swindon. 
Firm conclusions cannot realistically be 
drawn from one study. The 2003 pilots, 
which Swindon will be taking part in, will 
provide more data with which to assess the 
question of whether remote Internet voting 
can increase, or help to increase, turnout. 
This study does indicate that merely changing 
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how people vote cannot solve the question of 

voter apathy and a low turnout. As Diplock 

found, the inconvenience of the voting 
process is only a minor consideration for non- 
voters. Their primary concerns are that: 

e their vote will make little difference to the 
outcome, the parties are too 
homogenised; 

e there is no alternative to the main parties, 
none of which appeal to them; and 

« they have a lack of knowledge to make an 
informed choice about whom to vote for. 


Addressing these issues, for example through 
more political education and greater 
interaction between politicians and the public 
at all times, rather than just in the run-up to 
an election, might encourage more people to 
vote. Providing more convenient voting 
methods is only part of the solution to the 
problem of a low turnout. 


Notes 


1 The DTLR has since been divided into three separate 
departments: electoral reform issues are now the 
responsibility of the Lord Chancellor's Department 
(www.Icd.gov.uk). 

2 AUS non-partisan, non-profit, voters’ rights 
organisation. 
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Abstract 


The British Life and Internet Project was launched in 2002 to 
explore the way people use the Internet in Britain. It will 
survey people online throughout each year on a range of 
topics linked to the use of the Internet. In the first survey 
that was conducted, just before Christmas 2002, 
respondents were asked a series of questions about their 
history of involvement with the Internet and the nature of 
their online behaviour. The findings indicated that online 
technology was used overwhelmingly for sending and 
receiving e-mails and for obtaining news and information 
linked to work and hobbies. Such is the demand of the 
Internet on the time of users that many acknowledged 
reducing the amount of time they spend on other activities, 
such as watching television, reading newspapers, and even 
going out to do the shopping. The Internet is, for most of its 
users, an important interpersonal communications medium, 
used to stay in touch and gossip with family and friends. 
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Introduction 


The Internet is one of the fastest growing 
communications phenomena ever, with 
dramatic increases in online activity observed 
over the closing years of the twientieth 
century. While the telephone took 75 years 
to reach 50m users world-wide, and 
television took 13 years, the Internet took 
only four years (UNDP, 1999). NUA figures 
for May 2002 showed more than 580m 
Internet users in the world, with the greatest 
concentrations being found in Europe 
(186m), North America (183m) and the 
Asia/Pacific region (168m). NUA is a 
company that monitors surveys by a large 
number of market research companies in 
different countries (Norris, 2001). Although 
the use of different surveys limits the 
comparability of the data, these estimates 
have been confirmed by other sources world- 
wide, including government departments, 
media organisations and interest groups 
(Arunachalam, 1999). 

There are wide disparities between nations 
in the degree to which significant Internet 
penetration has been achieved. This “digital 
divide” has been linked to economic, political, 
social and cultural factors and has been found 
to occur within nations as well as between 
them (Norris, 2001). While the uptake of new 
communications technologies has displayed 
regional and socio-economic variations, the 
Internet has, nonetheless, emerged as a 
significant new medium through which 
individuals obtain information and 
entertainment, and conduct a range of 
transactions. 

Research among the online populations of 
the USA and Europe has revealed that, while 
many people search for information on the 
World Wide Web, the most popular 
application of the Internet is to send e-mails 
to family and friends (Pew, 2000; 
Eurobarometer, 2000). By 2000, nearly seven 
in ten (69 per cent) Internet users across 
Europe had e-mailed family, friends or 
colleagues in the last three months, while 
approaching one in two said they had 
searched for educational materials (47 per 
cent) or product information (47 per cent). 
Around one in seven said they had purchased 
items such as books (14 per cent) or CDs (14 
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per cent) online. Other uses included 
searching for job-related information, 
listening to music online, visiting government 
Web sites, watching TV channels and bidding 
in auctions. All this activity illustrates the 
range of applications that characterise 
Internet behaviour. 

While much publicity has centred on the 
role the Internet can play in relation to 
business and consumerism, for many of its 
users its application is focused much closer to 
home. For the great majority of Internet 
users, being online is important because it 
provides a convenient, on-tap access route to 
family and friends. In the USA, the most 
advanced Internet nation in the world, most 
Internet users acknowledge that e-mail has 
improved their contacts with family and 
friends. The guilt over not having made the 
effort to see a relative or friend for a long time 
could be appeased by sending them an e-mail. 
In fact e-mail was more convenient than 
having to take time out to visit. Through the 
Internet, contact with friends could be 
maintained without having to spend so much 
time talking to them. Further, it was easier to 
have unpleasant conversations or to speak 
one’s mind via e-mail and such messages 
could clear the air when relationships were 
strained (Pew, 2000). 

Other research has confirmed these survey 
reports about the nature of interpersonal 
relations online. Within a computer-mediated 
environment, the relative anonymity and 
emotional distance it creates can cause people 
to communicate with each other differently 
than from face-to-face meetings. We may all 
display greater bluntness of expression online 
and more honesty in the way we disclose our 
feelings (Joinson, 2001; Joinson and Banyard. 
2002; Postmes et al., 2002). 

As users become more experienced online, 
the Internet can become transformed into a 
purposeful tool with a range of applications 
(Pew, 2002a). From initially learning how to 
use e-mail to send messages to people we 
know, as user competence grows, the Internet 
is used for more complex applications 
including work-related tasks, to make 
purchases and to conduct financial 
transactions. It can be used to find 
information about a wide range of topics, 
either to satisfy idle curiosity or for more 
specific reasons. US research has indicated 
that the Internet can play crucial 
information-providing and advisory roles in 
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relation to a number of major life events. 
Users indicated that they used the Internet in 
relation to education and training for their 
career, getting a new job, obtaining 
information to help a loved one who was 
seriously ill, choosing the best available school 
for their children, and for making significant 
purchases such as a car (Pew, 2002b). 

In addition to the functions it can serve for 
individual users, the Internet has been 
promoted as enhancing democracy by making 
governments more accessible to citizens, 
providing transparent information about 
government activities and making those who 
govern more accountable. In Britain, the 
Government has set the ambitious target of 
ensuring that virtually all government 
departments — central and local — are online 
by 2005. Government Web sites will not 
simply provide information about government 
policies and activities, but will function as 
transactional links through which citizens can 
engage directly with a wide range of public 
services (Office of the E-envoy, 2001). As yet, 
while there has been significant growth in 
government Web sites, internationally, 
sophisticated interactive communications 
links and transactional services have been 
limited (Norris, 2001). Nonetheless, evidence 
has emerged that significant minorities of 
citizens have begun to use government and 
party political Web sites (Eurobarometer, 
2000). 

This paper reports initial findings from a 
new research project that has been designed 
to track the behaviour and opinions of 
Internet users in Britain. The British Life and 
Internet Project (BLIP) was launched in 2002 
to explore the way people use the Internet in 
Britain. Surveys are conducted online with 
known Internet users around the country[1]. 
The survey has access to some 30,000 UK 
e-mail addresses that have been compiled 
from a number of Internet user databases. 

Periodically, online questionnaires are 
posted on the BLIP Web site 
(www.britishlifeproject.co.uk) or e-mailed to 
members and Internet users invited to 
respond. Each survey collects information 
about the demographic characteristics of the 
sample and respondents’ Internet using 
history. In addition, surveys focus on specific 
themes that are concerned with the way 
people use the Internet, what they use it for, 
their opinions about Internet services and its 
impact on their lives. 
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The current survey, which served to launch 
the project, focused on the way people use the 
Internet around Christmas time. It 
investigated the way people use the Internet at 
this special time of year to keep in touch with 
family and friends and to seek out and 
purchase Christmas gifts. How much do 
Internet users now depend on cyberspace for 
their Christmas shopping? How much do they 
spend? Do they have any reservations or 
concerns about shopping on the Internet? 
How significant has the Internet become, 
alongside or in place of more traditional forms 
of Christmas-time communication, in 
enabling people to keep in touch with their 
loved ones? 


The Christmas Internet survey 


The fieldwork was conducted between 

2-10 December 2002. In all, 12,400 Internet 
users were contacted about the survey and 
2,887 responded (giving a 23 per cent 
response rate). The initial survey was sent to a 
database of users provided by The 
Independent Digital Group. 


Demographic profile 

The response sample comprised nearly two- 
thirds male respondents (64 per cent) and over 
one-third female respondents (36 per cent). 
The sample exhibited a diverse age spread: 
18-24 (14 per cent), 25-34 (25 per cent), 
35-44 (24 per cent), 45-54 (21 per cent), 


55-64 (11 per cent), and 65+ (5 per cent). 


More than two-thirds of respondents were 
in full-time work (69 per cent), with under 
one in ten, respectively, in part-time 
employment (9 per cent) or unemployed (9 
per cent). Just over one in ten (11 per cent) 
reported that they were in full-time 
education. 

One in three respondents were single (33 
per cent), more than four in ten (43 per cent) 
were married, and one in six (16 per cent) 
were living with a partner to whom they were 
not married. A small minority (8 per cent) 
were divorced, separated or widowed. 

A clear majority of respondents (75 per 
cent) had no children up to age 16 living with 
them in their household, one in eight (12 per 
cent) hed one such child, and one in seven 
(14 per cent) had two or more children living 
at home. 
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An overwhelming majority of respondents 
(92 per cent) were white and of European 
origin. The remainder were divided among 
Indian/Pakistani/Bangladeshi (2 per cent), 
Arab (1 per cent), Chinese (1 per cent), other 
Asian (1 per cent), Caribbean (1 per cent) 
and African (1 per cent) respondents. 


Newspaper readership 

Turning to daily newspaper readership, 
respondents were asked which daily or 
evening newspapers they personally read 
regularly (meaning at least two copies a 
week). The most commonly reported daily 
newspaper read by respondents was The 
Independent (26 per cent), followed by The 
Guardian (21 per cent). This is explained by 
the fact that the survey universe was 
constructed significantly from an e-mail 
database supplied by the Independent 
Newspaper Group. The next most popularly 
read daily newspapers were The Times (10 per 
cent) and The Daily Telegraph (7 per cent). 
Mid-range tabloid readers of newspapers such 
as the Daily Mail (3 per cent) and Daily 
Express (1 per cent) were much less 
commonplace among the sample, with 
Financial Times readers (4 per cent) being 
more frequently occurring. Readers of 
tabloids such as The Sun (2 per cent), The 
Mirror (2 per cent) and Daily Star (less than 
0.5 per cent) indicate the broadsheet bias in 
the sample. 

This pattern was reflected in claimed 
Sunday newspaper reading, for which regular 
reading meant at least two copies a month. 
The most read newspapers among this sample 
were The Observer (21 per cent), Independent 
on Sunday (21 per cent), The Sunday Times 
(17 per cent) and The Sunday Telegraph (7 per 
cent). These were followed by The Mail on 
Sunday (4 per cent), News of the World (3 per 
cent), Sunday Mirror (2 per cent), Sunday 
Express (1 per cent) and then The People, Star 
on Sunday all achieving fewer than 1 per cent 
of respondents saying they were regular 
readers. 


Internet history 

The overwhelming majority of respondents 
(91 per cent) had Internet access at home, 
with under one in ten being wholly dependent 
on gaining access elsewhere (9 per cent). 
Male respondents (94 per cent) were 
somewhat more likely than female 
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respondents (87 per cent) to say that they had 
Internet access at home. Well over half of 
respondents (58 per cent) indicated that they 
had had Internet access at home for at least 
three years. More than one in four (27 per 
cent) said they had had home Internet access 
for between one and three years. 

Male respondents (63 per cent) were more 
likely than female respondents (49 per cent) 
to report home Internet access for three or 
more years. There was an age difference in 
reported length of access to the Internet at 
home. The under-34s (42 per cent) were less 
likely than those aged 35 and over (69 per 
cent) to claim to have had home access to the 
Internet for 3+ years. Married respondents 
(68 per cent) were far more likely than either 
cohabiting respondents (53 per cent) or single 
respondents (44 per cent) to say they had had 
home Internet access for three or more years. 

Most respondents were regular users of the 
Internet. Nearly four in ten (39 per cent) of 
those with home access said they used the 
Internet from home at least twice a day, with 
more than one in five (22 per cent) saying 
they used the Internet at home at least once a 
day. More than one in four of these 
respondents (27 per cent) claimed to access 
the Internet from home at least once a week. 
Male respondents were somewhat more 
regular Internet users than female 
respondents. More than four in ten males (43 
per cent) reported using the Internet from 
home two or more times a day, compared 
with just under one in three females (32 per 
cent). The heaviest users of the Internet at 
home were the oldest respondents. Those 
aged 18 to 34 (23 per cent) claimed to use the 
Internet from home two or more times a day 
compared with 43 per cent of those aged 
35-54 and 64 per cent of those aged 55+. 

More than two-thirds of respondents (68 
per cent) said they accessed the Internet from 
work at least once a day, with a further one in 
ten (10 per cent) saying they did this at least 
once a week. Around one in six respondents 
(18 per cent) said they never accessed the 
Internet from work. 

No gender differences emerged in 
frequency of Internet access from work. It was 
not surprising, though, that the oldest age 
groups were most likely to say they never used 
the Internet at work. The over-65s (78 per 
cent) were most likely to say this, followed by 
a much smaller proportion of 55-64s (9 per 
cent) and an even smaller proportion of 
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under-55s (11 per cent). Hence, the age- 
related pattern of Internet use was clear. 
Older respondents used the Internet 
predominantly from home, whereas younger 
respondents. who were still in employment, 
did so mostly from their place of work. 
Marital status also exhibited a link to work 
use of the Internet. Married respondents (22 
per cent) were more likely than single 
respondents (12 per cent) or cohabiting 
respondents (8 per cent) to say they had never 
used the Internet at work. 

During completion of the online survey 
questionnaire itself, more than one in two 
respondents (53 per cent) said they did so 
from home, with more than four in ten (43 
per cent) reportedly doing so from work, and 
a tiny minority (3 per cent) doing so from an 
Internet café or from a friend’s home. 


General Internet-related activities 
Respondents were questioned about general 
Internet-related activities. A list of 37 
activities was provided and in each case 
respondents had to indicate whether they had 
done any of these things on the Internet in the 
past week, in the past month, in the past year, 
or whether they had never done these things 
online. 

The activities most often endorsed as 
having been engaged in online during the 
previous week were: 

* sending/reading e-mail (99 per cent); 

* getting news online (89 per cent); 

+ doing job-related work (69 per cent); 

* looking for information linked to a 
hobby/interest (65 per cent); . 

* looking for product-related information 
such as about books, toys, music (62 per 
cent); 

* going online just for fun or to pass time 
(59 per cent); 

* conducting personal banking online (49 
per cent); 

* checking weather forecasts (39 per cent); 

* getting travel information about 
airline/hotel prices (34 per cent); 

e checking sports scores (31 per cent); 

* visiting central government Web sites (30 
per cent); and 

* getting ideas for gifts to buy for Christmas 
for family/friends (27 per cent). 


Some age-related differences in the nature of 
use of the Internet emerged. The under 34s 
(84 per cent) were more likely than either the 
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35-54s (69 per cent) or the over 55s (49 per 
cent) to have ever visited a Web site 
associated with a television programme. 
While an overwhelming majority of under 55s 
(94 per cent) said they had, at some time, 
engaged in work-related activities online; 
fewer 55-64s (76 per cent) said this, while far 
fewer respondents aged over 65 (47 per cent) 
had reportedly ever done this. Instant 
messaging online was an activity most often 
associated with those aged 24 and under (70 
per cent of whom had ever done this) and 
least often associated with the over 55s (46 
per cent). Yet there were no significant age 
differences in the extent to which respondents 
claimed to have sent online Christmas or 
birthday greetings to family members. The 
most likely users of adult Web sites were 
respondents aged 35-44 years. This was the 
only age group for which the majority of 
respondents (53 per cent) claimed ever to 
have visited one of these sites. 

Single respondents (50 per cent) and 
cohabiting respondents (45 per cent) were 
much more likely than married respondents 
(30 per cent) to say they had ever taken part 
in chat rooms online. Single respondents (61 
per cent) were also more likely than both 
married (42 per cent) and cohabiting 
respondents (51 per cent) to have ever sent 
instant messages to someone online at the 
same time. Despite the social usage of the 
Internet, only a minority overall used it to find 
a date. This use of the Internet was more 
prevalent among single respondents (18 per 
cent) than among married respondents (3 per 
cent), though cohabiting respondents were 
markedly more likely than their married 
counterparts to report having sought an 
online date (11 per cent). 


Purchasing via the Internet 

There were signs that Internet users are using 
the Internet not only to get ideas about things 
to buy for themselves or for others, but also to 
make those actual purchases. For instance, 
eight in ten respondents (80 per cent) had 
purchased a book online within the past year, 
with more than one in five (21 per cent) 
saying they had done so within the previous 
week. 

Again, eight in ten respondents (80 per 
cent) said that they had bought or made a 
reservation for travel online within the past 
year, with one in six (16 per cent) claiming to 
have done so within the previous week. 
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More than one in two respondents (51 per 
cent) said they had made online purchases of 
groceries or household items over the past 
year, with nine per cent reportedly having 
done this within the past week. 


Interpersonal communication and the Internet 
The Internet was also used as a 
communications medium for keeping in 
touch with others, whether relatives, friends 
or strangers. More than one in ten 
respondents (11 per cent) claimed to have 
used the Internet within the past year to find 
someone to date. Nearly one in four (24 per 
cent) said they had used the Internet within 
the past year to meet new people and make 
new friends. A total of four in ten (40 per 
cent) had, within the past year, taken part in 
“chat rooms” or online discussions with other 
people. Ethnic minority respondents (53 per 
cent) were less likely to report joining in chat 
rooms than were Caucasian respondents (61 
per cent). More than one in two (51 per cent) 
had, within the last year, sent “instant 
messages” to someone who was online at the 
same time. Finally, seven in ten respondents 
had used the Internet at some point within the 
past year to send online Christmas or birthday 
greetings to family and friends. And, of 
course, as already noted above, nearly all 
respondents (99 per cent+) had used the 
Internet for e-mail purposes. 


Online democracy 

The government has announced ambitious 
plans to establish online access to central and 
local government universally by 2005. The 
current research indicates that Internet users 
in Britain are already making use of such 
online services. More than nine in ten 
respondents (91 per cent) claimed to have 
visited a central government Web site in the 
past year, with three in ten (30 per cent) 
claiming to have done so within the previous 
week. A total of eight in ten respondents (80 
per cent) had reportedly visited local 
government Web sites for information about 
local public services during the past year, with 
one in five (20 per cent) saying they had done 
this in the past week. A smaller proportion, 
but still over half of respondents (54 per 
cent), claimed to have visited political party 
Web sites in the past year, with one in ten (10 
per cent) saying they had done this in the past 
week. 
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Internet as an information source 

The evidence from this survey indicated 
clearly that the Internet is an important 
information source for its users. Virtually all 
respondents (99 per cent) had used the 
Internet to get news, and a significant 
majority (87 per cent) had used it for weather 
reports. More than six in ten respondents (63 
per cent) had at some time checked online 
sports scores and over nine in ten (95 per 
cent) had accessed online travel information 
in the past year. 

Male respondents (41 per cent) were 
much more likely than female respondents 
(12 per cent) to say they had checked sports 
scores on the Internet during the previous 
week. Female respondents (54 per cent) 
were much more likely than males (28 per 
cent) to say that this was something they 
never did. In addition, respondents with two 
or more children at home (40 per cent) were 
more likely to have checked sports results on 
the Web in the previous week than were 
either respondents with just one child at 
home (30 per cent) or no children at home 
(29 per cent). 

On a more personal front, virtually all 
respondents (99 per cent) had accessed online 
information relating to a hobby or interest 
within the last year, and nearly six in ten (58 
per cent) had gone online at some point for 
housing information. 

Underlining the growing importance of the 
Internet to consumers, most respondents (98 
per cent) had, at some point in the previous 
year, looked for product-related information 
online. Significantly, and probably linked ta 
the proximity to Christmas, more than six in 
ten (62 per cent) said they had looked for 
product-related information online within the 
past week. 


Internet as entertainment 
The Internet is not just a source of 
information, it is also a source of 
entertainment for its converts. While 
relatively few respondents (9 per cent) 
claimed to have watched a movie online, most 
(77 per cent) had watched video clips online 
within the past year. Male respondents (24 
per cent) were more likely than female 
respondents (13 per cent) to say they had 
watched a video clip online during the 
previous week. 

In a hybrid entertainment-related 
informational application of the Internet, 
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more than seven in ten respondents.(72 per 
cent) said they had visited Web sites linked to 
TV shows within the previous 12 months. 
At the more risqué end of usage, nearly one 
in two respondents (47 per cent) claimed to 
have visited an adult Web site in the past year. 
Males (15 per cent) were significantly more 
likely than females (1 per cent) to say they had 
visited one of these sites in the past week. 
Female respondents (83 per cent), 
meanwhile, were much more likely than male 
respondents (37 per cent) to say they never 
visited adult Web sites on the Internet. 


Financial usage 

Some evidence emerged that many Internet 
users utilise online financial services. More 
than seven in ten respondents (71 per cent) 
said they had conducted personal banking 
online in the past 12 months, with nearly one 
in two (49 per cent) having done so within the 
previous week. Male respondents (53 per 
cent) were somewhat more likely than female 
respondents to say they had conducted online 
banking during the past week. Only a 
minority (12 per cent) had bought or sold 
stocks and shares online, however, over the 
previous year. 


Downloading content 

The Internet is not just a source of online 
information and entertainment for live 
consumption. It provides users with access to 
content they can download for re-use. More 
than eight in ten respondents {81 per cent) 
indicated that they had ever downloaded 
software online relating to music or digital 
photography. More than one in five (21 per 
cent) said they had done this during the 
previous week. This recent activity was 
reportedly more prevalent among male 
respondents (26 per cent) than female 
respondents (11 per cent). 

A clear majority of respondents (64 per 
cent) also claimed ever to have downloaded 
other kinds of files such as games, video files 
or picture files. Far fewer respondents 
reported ever having downloaded movies (16 
per cent) or music recordings (38 per cent), 
or to have shared files from their own 
computer such as music, video or picture 
files, or computer games with others online 
(30 per cent). Male respondents (34 per cent) 
were more likely ever to have engaged in file 
sharing online than were female respondents 
(22 per cent). 
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Those activities most often endorsed as 

never having been done online were: 

* watching a movie online (91 per cent); 

* finding someone to date (89 per cent); 

* playing a lottery or gambling online (89 
per cent); 

* buying stocks and shares online (88 per 
cent’; 

+ downloading movies (86 per cent); 

* meeting new people or making new 
friends (76 per cent); 

* participating in online auctions (74 per 
cent}; 

* getting photos developed/storing/ 
displaying photos online (72 per cent); 

«sharing files from own computer (e.g. 
music, video, picture files) (70 per cent); 
and 

* obtaining music recordings online (62 per 
cent). 


Internet and use of time 

Survey respondents were asked whether using 
the Internet had changed the amount of time 
that they spend on a variety of other activities. 
Had the Internet increased or decreased the 
amount of time spent on the listed activities? 
A total of 11 activities were presented in this 
context: working at the office; working at 
home; shopping in stores; commuting in 
traffic; reading newspapers; watching 
television; spending time with my family; 
thinking about what to buy people for 
Christmas; spending time with my friends, 
and attending social events and activities. 

In terms of percentages of respondents 
saying that using the Internet had decreased 
the amount of time they spend on each of 
these activities, the ranking that emerged is as 
follows: 

e watching television (42 per cent); 

* reading newspapers (36 per cent); 

* going around shops buying Christmas 
presents (34 per cent); 

* shopping in stores (33 per cent); 

* commuting in traffic (16 per cent); 

* working in the office (13 per cent); 

* thinking about what to buy people for 
Christmas (11 per cent); 

* spending time with my family (11 per 
cent); 

* working at home (6 per cent); 

* spending time with my friends (5 per 
cent); and 


Volume 55 - Number 4 - 2003 - 203-216 


* attending social events and activities (5 
per cent). 


Internet use appears, on the basis of this 
self-report evidence, to have had the greatest 
impact on watching television and reading 
newspapers. While the greatest proportion of 
respondents said that using the Internet had 
made no difference to time spent watching 
television (56 per cent) or reading newspapers 
(50 per cent), as the figures above show, 
significant minorities of respondents said that 
this new medium had led them to reduce the 
amount of time they spend consuming these 
two longer-established media. 

The one major gender difference in relation 
to the reported impact of using the Internet 
on other activities emerged in relation to 
television viewing. Here, male respondents 
(47 per cent) were more likely than female 
respondents (33 per cent) to say that using 
the Internet had decreased the amount of 
time they watched television. 

For most respondents, the Internet made 
no difference to the amount of time they spent 
working at the office (73 per cent) or at home 
(54 per cent). In regard to working at the 
office, the reported impact of the Internet was 
fairly evenly divided between those saying it 
had increased the time they spent at the office 
(14 per cent) or decreased it (13 per cent). 
Where a clearer impact was becoming 
established, however, was in respect of 
working from home. Here, many more 
respondents said that use of the Internet had 
resulted in increased amount of time spent 
working at home (40 per cent) than decreased 
time working at home (6 per cent). This 
finding probably also links with reports that 
the use of the Internet had decreased (16 per 
cent) rather than increased (1 per cent) time 
spent in traffic commuting. Number of 
children in the household was associated with 
the extent to which respondents claimed to 
have increased the amount of time they spend 
working at home. Respondents with two or 
more children at home (51 per cent) were 
more likely than respondents with one child 
(41 per cent) or no children at home (38 per 
cent) to say that using the Internet had 
increased the time they spent working at 
home. 

Respondents aged 25-44 (40 per cent) were 
most likely to claim that using the Internet 
had decreased the amount of time they spent 
going round the shops buying Christmas 
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presents, with the 45 and over age groups (30 
per cent) being less likely to admit to this 
change, and the under 24s (25 per cent) least 
likely of all age groups to acknowledge this 
effect of the Internet. 

The social impact of the Internet was 
marginal. Most respondents indicated that 
using the Internet had not changed the 
amount of time they spent with their family 
(86 per cent) or friends (90 per cent) or going 
to social events (89 per cent). 

The use of the Internet at Christmas for 
shopping and buying presents was in 
evidence. One in three respondents (33 per 
cent) said the Internet had decreased the 
amount of time spent shopping in stores, with 
few (4 per cent) saying the Internet had 
increased store shopping time. Reinforcing 
this last finding, more than one in three 
respondents (34 per cent) also said that using 
the Internet had decreased (compared with 2 
per cent saying increased) time spent going 
around shops buying Christmas presents. 
Opinions were more mixed on whether the 
Internet saved time thinking about what to 
buy people for Christmas, with 11 per cent 
saying it decreased time spent doing this and 
13 per cent saying it increased time spent 
on it. 


Importance of e-mail to its users 

Over nine in ten respondents (92 per cent) 
claimed to have sent at least one e-mail 
“yesterday” (i.e. the day before they 
responded to the survey). One in four 
respondents (25 per cent) claimed to have 
sent three to five e-mails, one in three (33 per 
cent) said they had sent between six and 20 
e-mails, and one in seven (14 per cent) had 
reportedly sent 21 or more e-mails the 
previous day. Respondents aged 24 to 35 (44 
per cent) were most likely to have sent 11 or 
more e-mails during the previous day, with 
the youngest (18-24: 15 per cent) and oldest 
(65+: 11 per cent) being the least likely to 
have sent this many. 

When asked how much they would miss 
using e-mail if it was no longer available to 
them, three out of four respondents (76 per 
cent) said they would miss it “a lot”. Most of 
the remaining respondents (17 per cent) said 
they would miss e-mail “quite a bit”. 

Reporting on their behaviour over the past 
week, most respondents said they had 
e-mailed a friend (88 per cent) or a relative 
(60 per cent). This was comparable with 
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telephoning a friend or relative (82 pér cent), 
or visiting with friends (73 per cent), but 
compared favourably with visiting with family 
(50 per cent), having friends over to my home 
(46 per cent), and having family over to my 
home (28 per cent). The oldest respondents 
were most likely to have e-mailed a member 
of their family in the past week. A clear 
majority of over 55s (77 per cent) said they 
e-mailed a relative in the previous week, with 
younger age groups (56 per cent) being much 
less likely to have done so. 

Most respondents said they found e-mail 
either “very useful” (42 per cent) or “quite 
useful” (31 per cent) for communicating with 
members of their family. Even more found 
e-mail “very useful” (67 per cent) for keeping 
in touch with friends, with many others (26 
per cent) finding it “quite useful” for this 
purpose. Older respondents aged 55+ (57 per 
cent) were more likely than younger 
respondents (under 55: 38 per cent) to say 
they found e-mail “very useful” for 
communicating with family members. For the 
youngest respondents, aged up to 35 (77 per 
cent), e-mail was “very useful” for 
communicating with friends, with older 
respondents (61 per cent) being somewhat 
less likely to say this. 


E-mailing family 

Respondents were asked to indicate which 
member of their immediate or extended 
family they e-mail most often. Just over one in 
ten (11 per cent) claimed never to e-mail 
family members. Among the remainder of the 
sample, the family members e-mailed most 
often were ranked as follows: 

* sister (15 per cent); 

+ brother (13 per cent); 

* wife (9 per cent); 

* mother (9 per cent); 

* daughter (8 per cent); 

-+ father (7 per cent); 

* son (7 per cent); 

- husband (5 per cent); 


.* cousin (5 per cent); 


* niece/nephew (2 per cent); 

* in-law (2 per cent); 

*  aunt/uncle (2 per cent); 

+ grandchild (less than 1 per cent); and 
¢ grandparent (less than 1 per cent). 


E-mailing was found to occur most often 
between siblings (28 per cent), followed by 
with parents (16 per cent), and then with 
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children (15 per cent) or between spouses (15 
per cent). 

When respondents were asked to provide 
further information about the person e-mailed 
most often in terms of where they lived, it 
became clear that this person tended to live at 
a locaticn remote from the respondents. In 
nearly four in ten cases, this person lived in a 
different part of the country (39 per cent) and 


in more than one in four cases (26 per cent) ~ 


overseas in a different country. It is interesting 

to note, nonetheless, that in nearly one in five 

cases (19 per cent) the family member 
e-mailed most often lived in the respondent’s 

home, and in a similar proportion of cases (17 

per cent) they lived in the same community or 

area. 

The Internet may provide a substitute for 
face-to-face contact with the family member 
e-mailed most often. For most respondents 
(57 per cent) they reported actually seeing 
this person just a few times a year or less 
often. At the other extreme, there are family 
members with whom regular face-to-face 
contact does not diminish e-mailing them too. 
In nearly one in five cases, respondents saw 
the family member they e-mailed most often 
every day. Significantly, the family member 
most often e-mailed may also be contacted by 
telephone fairly frequently too. In more than 
one in two cases (52 per cent), this person was 
reportedly telephoned at least once a week. 

Getting respondents to think further about 
the relative they most often e-mailed, they 
were also asked to indicate the kinds of things 
they e-mailed this person about in the past 
four weeks. Ten different reasons for 
e-mailing were provided. These are shown 
below in rank order according to how often 
they were selected by respondents. The 
relative most often e-mailed was contacted 
online in the past four weeks: 

* to pass along something interesting you 
heard or read (76 per cent); 

* to pass along some news about yourself, 
family or people you know (73 per cent); 

* to pass along a joke or something funny 
you’ve heard (58 per cent); 

* to pass along something about your 
job/other things involved with (53 per 
cent); 

¢ about getting together for dinner or social 
activity (40 per cent); 

* to get advice about something (38 per 
cent); 
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e to make arrangements for Christmas (35 
per cent); 

* to tell them about something that has 
upset or worried you (32 per cent); 

+ to find out what they want as a present for 
Christmas (25 per cent); and 

* to wish them a merry Christmas (8 per 
cent). 


Thus, e-mail is used primarily to pass along 
and share information with family, either 
about self, family matters or other things of 
mutual interest. It is still significant, however, 
that more than one in three respondents used 
e-mail to make arrangements for Christmas 
and one in four used it to find out what the 
other person might want as a Christmas gift. 

Some gender-related patterns of use 
emerged in relation to the Internet as a social 
activity. Female respondents were more likely 
than male respondents to say they e-mailed a 
family member in the past four weeks to pass 
along a joke or something funny they had 
heard (66 per cent versus 54 per cent), to tell 
them about something that had upset them 
(40 per cent versus 27 per cent), and to pass 
along something about their job or other 
things they had been involved in (59 per cent 
versus 50 per cent). 

The youngest respondents, aged up to 35 
(64 per cent), were most likely to have used 
e-mail in the past month to pass along a joke 
or funny story. Older age groups (52 per cent) 
were somewhat less likely to use e-mail for 
this purpose. The over 55s (14 per cent) were 
twice as likely as the under 45s (7 per cent) to 
report using e-mail to wish a relative a merry 
Christmas. 


E-mailing friends 

Respondents were asked about e-mailing 
friends as well as family. When asked which of 
their friends they e-mailed most often — a 
friend who lives close by or one who lives far 
away, more than six in ten (62 per cent) said 
they e-mailed a geographically remote friend. 

In nearly six in ten cases (59 per cent), the 
friend e-mailed most often lived either in 
another part of the country (30 per cent) or in 
a different country (29 per cent). In nearly 
four in ten cases (39 per cent) the friend lived 
in the same community or area. 

E-mailing a friend was, to some extent, a 
substitute for seeing them face to face. It was 
also a partial substitute for telephoning them. 
In nearly six in ten cases (58 per cent) the 
respondent saw their friend a few times a year 
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at most. In one in two cases (50 per cent), to 
one in eight cases did they see this friend (12 
per cent) or telephone them (14 per cent) a 
few times a week or more. 

Respondents were also asked to give the 
most usual reasons for e-mailing this friend. 
The same ten reasons were provided as for 
family members. Thus, over the previous four 
weeks, respondents claimed to have e-mailed 
their friend: 

(1) to pass along something interesting they 
had heard or read (79 per cent); 

(2) to pass along news about yourself, family 
or people you know (76 per cent); 

(3) to pass along a joke or something funny 
you have heard (66 per cent); 

(4) to pass along something about your 
job/other things involved in (62 per 
cent); 

(5) about getting together for dinner or 
social activity (54 per cent); 

(6) to get advice about something (41 per 
cent); 

(7) to tell them about something that has 
upset or worried you (37 per cent); 

(8) to arrange to see them over Christmas 
(23 per cent); 

(9) to wish them a merry Christmas (13 per 
cent); and 

(10) to find out what to buy them for 
Christmas (9 per cent). 
The pattern with e-mailing friends was similar 
to that for e-mailing family. E-mail is used 
mostly to pass along information and 
experiences of mutual interest. A few 
respondents did refer to using e-mail to make 
Christmas arrangements with friends, but this 
form of communication was less popular in 
this context in relation to friends than in 
relation to family. As with e-mailing relatives, 
e-mailing friends exhibited gender-related 
differences. Female respondents were more 
likely than male respondents to say they had 
e-mailed a friend in the past four weeks to tell 
them about something that had upset or 
worried them (47 per cent versus 30 per cent) 
and to pass along some news about 
themselves, their family or people they knew 
(85 per cent versus 71 per cent). E-mail is a 
medium for gossip. 

The youngest respondents (18-34s: 67 per 
cent) were most likely to use e-mail to make 
social arrangements with friends. This age 
band was also most likely to have used e-mail 
to pass along a joke to a friend (44 per cent) 
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or to gossip about work or other events in 
their lives (67 per cent). 

Single respondents (66 per cent) and 
cohabiting respondents (63 per cent) were 
more likely to use e-mail for making social 
arrangements than were married respondents 
(43 per cent). Single respondents (69 per cent) 
and cohabiting respondents (65 per cent) were 
also more likely to have e-mailed a friend in the 
past four weeks to pass along something about 
their job or other events in their lives than were 
married respondents (55 per cent). 


The Internet and Christmas shopping 
One of the principal themes of this survey was 
to find out the extent to which the Internet is 
used these days for Christmas shopping. To 
what extent do Internet users use it to get 
ideas for gifts and to make actual purchases? 

To begin with the survey obtained some 
wider background from respondents about 
Christmas shopping. How many people did 
they have to buy gifts for? Who were these 
people? How much money were they likely to 
spend on buying Christmas gifts? 

More than one in three respondents (36 per 
cent) said they had to buy Christmas gifts for 
ten or more people. More than four in ten 
respondents (42 per cent) had to buy gifts for 
between five and nine people, while just 
under one in five (19 per cent) had to buy 
gifts for four or fewer people. Just a few 
respondents (4 per cent) said they had no one 
they needed to buy gifts for. Female 
respondents (45 per cent) were more likely 
than male respondents (31 per cent) to say 
they had to buy Christmas gifts for ten or 
more people. Not surprisingly, respondents 
with either two or more (46 per cent) children 
or one child in the home (43 per cent) were 
more likely than respondents with no children 
living at home (33 per cent) to say they had to 
buy Christmas gifts for ten or more people. 

When asked which people Christmas gifts 
would have to be purchased for, respondents 
answered: 

*  mother/father (19 per cent); 

* other relative (17 per cent); 

* friend (16 per cent); 

*  spouse/partner (15 per cent); 

* son/daughter (11 per cent); 

e work colleague (6 per cent); 

* boyfriend/girlfriend (5 per cent); 
* grandparent (4 per cent); 

* grandchild (2 per cent); and 

* other (5 per cent). 
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In total, most respondents {62 per cent) 
expected to spend between £100-500 buying 
Christmas gifts, with one in seven (15 per 
cent) expecting to spend more than this, and 
over one in five (22 per cent) expecting to 
spend less. For most respondents (62 per 
cent) the most expensive single gift they 
expected to buy would cost between 
£21-100, for one in six (16 per cent) it was 
likely to cost £100-200, and for a small 
minority (7 per cent) more than £200. For 
one in eight (12 per cent) the most expensive 
gift was expected to cost no more than £20. 

Whether or not respondents had children 
living at home with them made a difference to 
how much money they expected to spend on 
Christmas gifts in 2002. Respondents with 
two or more children at home (70 per cent) or 
one child at home (67 per cent) were more 
likely to say they would spend £200 or more 
on Christmas gifts as were respondents with 
no children living at home with them (47 per 
cent). 

Male respondents (5 per cent) were two 
and a half times as likely as female 
respondents (2 per cent) to say they would 
spend more than £1,000 in total on buying 
Christmas gifts this year. 

Married respondents (45 per cent) were the 
most likely to say they expected to spend 
£300 or more on buying Christmas gifts, with 
cohabiting respondents (34 per cent) and 
single respondents (14 per cent) being 
progressively less likely to say they would 
spend this amount. 

When asked whether they had used the 
Internet yet this year to find out information 
or to get ideas about Christmas gifts, more 
than six in ten respondents (61 per cent) said 
that they had done so. Among those (39 per 
cent) wko said they had not used the Internet 
in this way yet, between one in five and one in 
four (23 per cent) said they expected to do so. 
Thus, among these Internet users, if they had 
not yet used the Internet to get Christmas gift 
ideas (during the first half of December), 
most (77 per cent) were unlikely to do so. 

When respondents were asked whether they 
had used the Internet to purchase any 
Christmas gifts this year, nearly six in ten (58 
per cent} said they had done so. Among those 
(42 per cent) who had not done this, most (58 
per cent) said that they would do so 
eventually. Married (63 per cent) and 
cohabiting respondents (63 per cent) were 
more likely than single respondents (49 per 
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cent) to say they had used the Internet this 
year to purchase any Christmas gifts. 
Among respondents who had decided not 
to purchase gifts on the Internet, they were 
asked to give reasons for not doing so. A total 
of 11 reasons were provided and respondents 
were invited to say how important each of 
these reasons was. In terms of percentages of 
respondents saying these reasons were “very 
important”, the findings revealed the 
following: 
(1) I want to see the items I purchase (49 
per cent); 
(2) I feel there are better ways to shop (32 
per cent); 
(3) don’t want to risk using my credit card 
online (19 per cent); 
(4) not interested in shopping online (19 per 
cent); 
(5) concerned about quality of merchandise 
(18 per cent); 
(6) don’t have a credit card (9 per cent); 
(7) not buying gifts this year (6 per cent); 
(8) not comfortable using a computer or 
going to shopping Web sites (5 per 
cent); 
(9) don’t have time to shop online (3 per 
cent); 
(10) tried shopping online and it didn’t work 
(3 per cent); and 
(11) don’t have easy access to the Internet (2 
per cent). 
The findings indicate that among those 
respondents who had not purchased 
Christmas gifts online and did not intend 
doing so (17 per cent of total sample), the 
main reasons given were the need still to see 
the merchandise and belief that offline 
shopping was a better way to shop. Female 
respondents (55 per cent) were somewhat 
more likely than male respondents (45 per 
cent) to say they wanted to see the items they 
purchase. A minority also exhibited some 
concern about credit card security and quality 
of the merchandise purchased online. 
Respondents (58 per cent) who said they 
had purchased Christmas gifts online were 
asked how many such gifts they had bought in 
this way. One in five (20 per cent) said they 
had bought just one gift-online, one in four 
(26 per cent) had bought two gifts, over one 
in three (35 per cent) had purchased between 
three and five gifts, and nearly one in five (19 
per cent) had purchased six or more gifts 
online. Respondents with two or more 
children living at home with them (29 per 


213 


The British Life and internet Project: inaugural survey findings 


Aslib Proceedings 





Barrie Gunter, Chris Russell, Richard Withey and David Nicholas 


cent) were the most likely to say they had 
bought six or more gifts online, compared 
with 23 per cent of respondents with one 
child at home and 17 per cent of respondents 
with no children living with them. 

Among those respondents who had bought 
gifts online, more than half (57 per cent) had 
spent between £21 and £100, just under one 
in five (18 per cent) had spent £101-200, one 
in seven (15 per cent) had spent over £200, 
and one in ten (10 per cent) had spent up to 
£20. The same respondents were asked to say 
what kinds of gifts they had purchased online. 
The key findings were as follows: 

* books (26 per cent); 

* computer products (8 per cent); 

* children’s toys (8 per cent); 

* clothing (7 per cent); 

* hi-fi equipment (6 per cent); 

* video games (5 per cent); 

* small household appliances (toaster, 
kettle, blender, etc.) (3 per cent); 

*  ornaments/olyet d’arts (3 per cent); 

* video camera (3 per cent); 

* sports equipment (2 per cent); 

* children’s games (2 per cent); 

* ‘TV set/VCR (2 per cent); and 

* various other items — perfume/cosmetics, 
suitcases, jewellery, handbags, garden 
plants, garden ornaments, large 
household appliances (all under 2 per 
cent). 


Respondents with two or more children living 
at home were most likely to report buying 
children’s toys on the Internet, followed by 
respondents with one child at home (12 per 
cent) and respondents with no children living 
with them (6 per cent). Married respondents 
(10 per cent) were more likely than either 
single (5 per cent) or cohabiting respondents 
(6 per cent) to have bought children’s toys on 
the Internet. 

Using the Internet to make these purchases 
did not cause respondents any discomfort. 
The great majority (86 per cent) said they 
were “very comfortable” (34 per cent) or 
“quite comfortable” (52 per cent) giving their 
credit card details over the Internet. The over 
65s (17 per cent) were significantly less likely 
than average to say they felt “very 
comfortable” about giving their credit card 
details over the Internet. 

Finally, all respondents were asked about 
the different ways in which they might 
communicate with others this Christmas. A 
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clear majority, in each case, said they would 
mail letters or Christmas cards to family and 
friends (84 per cent), phone a family member 
to talk about Christmas holiday plans and 
events (82 per cent), and phone a friend to 
talk about Christmas-related plans (77 per 
cent). Hence all these Internet-linked 
respondents expected to use traditional forms 
of communication to reach out to family and 
friends at Christmas. 

More than half (57 per cent) also said they 
would send e-mails to family members to talk 
to them about Christmas events and to make 
plans, nearly one in two (49 per cent) also 
said they would send a Christmas card by 
e-mail. Interestingly, ethnic minority 
respondents (55 per cent) were 
proportionately more likely to say they would 
send a Christmas card by e-mail than were 
Caucasian respondents (48 per cent). Finally, 
over four in ten (42 per cent) said they would 
send an e-mail to someone they had not 
spoken to for a long time to exchange 
Christmas greetings. Such findings indicate 
the growing significance of the Internet at 
Christmas time as a channel through which 
people can interact with those they have not 
seen in ages. 


Discussion 


The survey of Internet users that has been 
reported here confirmed earlier research in 
the USA (Pew, 2002a, b) and in Europe 
(Eurobarometer, 2000) in showing that 
e-mail is the most widespread single 
application of the Internet. But while over 
two-thirds of Internet users said they had 
e-mailed family, friends or colleagues in the 
past three months, in the current survey, 
virtually all respondents (99 per cent) had 
used e-mail during the past week. Although 
the authors of this report cannot claim that 
the survey sample is fully representative of 
Internet users in Britain, this finding 
nonetheless underlines the significance of the 
Internet as a two-way communications 
medium. 

The prevalence of e-mail use may, of 
course, be explained in part by the timing of 
this survey. It is possible that use of the 
Internet to contact family and friends is 
especially likely to occur at Christmas time. 
For example, while Internet use has been 
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associated more with younger people, more 
than one in ten (11 per cent) respondents 
aged over 65 years claimed to have sent 11 or 
more e-mails the previous day. E-mail was 
used more often than the telephone to contact 
friends, though less often to contact relatives. 
In fact the significance of the Internet as a 
communications medium at Christmas was 
further underlined by marked age differences, 
with a larger proportion of over 55s (77 per 
cent) than of younger age groups (56 per 
cent) claiming to have e-mailed family 
members during the previous week. 

The current survey indicated a key 
difference in patterns of Internet use by age. 
The Internet is not the preserve of young 
people; many older respondents reported 
regular and long-term use of this medium. 
The significant difference between younger 
and older users was the primary location of 
online behaviour. For younger ones still at 
work, the Internet was accessed most often 
from the office, while for older respondents, it 
was used most often at home. 

As an interpersonal communications 
medium, the Internet is used first of all to 
gossip, then to share jokes, and then to make 
social arrangements. It is also used to get 
advice or to seek emotional support. More 
than one in three respondents used e-mail to 
make arrangements to see family members 
over Christmas, with nearly one in four 
using it to make such arrangements with 
friends. 

Another prominent feature of 
Internet-related behaviour for this online 
sample was the use of the Net for news 
information. An overwhelming majority (89 
per cent) claimed to have used the Internet 
over the previous week for that purpose. This 
finding is not surprising, however, given that 
the sample was derived from a population of 
online news subscribers. In fact, the current 
sample comprised many experienced 
Internet users, with well over half the 
respondents claiming to have had Internet 
access for at least three years. Job-related 
usage was widespread, but so too was looking 
for information linked to a hobby or interest, 
or simply going online for fun or to pass 
time. 

The diversity of Internet-related behaviour 
was further illustrated by the majority of 
respondents who went online to look for 
product-related information, travel 
information and to conduct financial 
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transactions. This type of use indicates the 
growing competence and confidence of 
Internet users in Britain. Most respondents 
(80 per cent) had purchased a book online in 
the past year, with the same proportion 
saying they had made an online travel 
reservation during that time. In addition, 
over half (51 per cent) had purchased 
groceries online. 

On current evidence, the Internet is 
emerging as a Christmas shopping medium. It 
was clear that many respondents had 
Christmas gifts to buy for a number of people. 
The majority of respondents in this survey 
claimed to have used the Internet to seek 
Christmas gift ideas and information and to 
purchase gifts. Furthermore, the great 
majority of respondents (80 per cent) who 
said they had purchased Christmas gifts over 
the Internet had bought more than one item 
this way. More than four in ten (43 per cent) 
spent more than £100 on gifts purchased 
online. All this shows the growing significance 
of the Internet as a consumer medium. 
Despite the difficulties experienced by early 
dot-com operators in the late 1990s, these 
latest findings offer encouraging signs that the 
Internet could evolve into an important 
consumer platform in the future. 


Note 


1 The British Life and Internet Project comprises a 
consortium consisting of researchers from the 
University of Sheffield and City University, 
eDigital Research and the Independent Newspaper 
Group. 
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Introduction 


It is argued here that open source software 
development is a promising method for 
producing information retrieval (IR) systems. 
Open source software is available freely 
(usually on the Web), and the source code is 
available so that changes may be made as 
required. Software which enjoys wide use has 
the most to gain from open source, and IR 
systems are likely to be increasingly used as 
more and more people and the organisations 
they work for require Web or intranet search. 
The paper is organised as follows. In order to 
give the reader some background in the area 
of open source software a description of this 
type of development is given. Then some of 
the systems that are mature and available for 
use by IR researchers and practitioners are 
outlined. An argument is provided for open 
source IR systems, followed by an outline of 
the potential problems and obstacles that may 
counter the benefits of open source in IR. 
Finally, an agenda is set for open source IR 
software in the conclusion. 


Open source software 


Open source software (Feller and Fitzgerald, 
2002) is where the source code of programs is 
made freely available for anyone to change 
and distribute providing they abide by the 
accompanying licence. This differs from 
closed source or propriety software which 
may only be obtained by some form of 
payment, either by purchase or by “leasing”. 
The difference between open and closed 
source can be characterised by the word 
freedom: users of open source software have 
the freedom to alter the source code while 
users of closed source software do not. 
There are many licences available for use 
with open source software (Feller and 
Fitzgerald, 2002), but the most popular of 
these is the GNU Public Licence (GPL). The 
GPL gives programmers the right to alter 
software and re-distribute it, providing that 
the changes they have made are available to 
other programmers: this concept is known as 
copyleft. Thus the GPL is said to be “viral” 
(Feller and Fitzgerald, 2002): any software 
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which uses the GPL must itself be released 
under the GPL. The reasoning behind 
copyleft is to prevent propriety software 
houses from exploiting the (often unpaid) 
work of open source programmers, without 
some recompense. 


Systems currently available 


There is a wide range of open source IR 
systems available on the Web from sites such 
as sourceforge.net and freshmeat.net. These 
systems can be categorised into two main 
groups: those that use inverted files and those 
that use database systems. We provide a short 
description of a small number of systems, 
which are mature or have been used by either 
IR practitioners or researchers and which 
used the inverted file approach. All of these 
systems are free software and most are 
released under the GNU Public Licence 
(GPL). The systems we concentrate on are 
Xapian, Swisht++, Senga, ht://Dig, Isearch 
Oasis, MG and Lemur (other freely available 
IR systems found in the course of our 
investigation are listed in the Appendix). 


Xapian 
(http://sourceforge.net/projects/xapian/) 
Xapian has had something of a chequered 
history. It was originally developed under the 
name of Open Muscat (Porter and Boulton, 
2000) by BrightStation PLC, changed its 
name to Omsee and then to OmSeek when 
the BrightStation open source project was 
closed down (due to financial problems 
unrelated to the project itself), and then to 
Xapian. However, it continues to be 
developed under the GNU Public Licence 
(GPL). 

Xapian is not a program but a suite of 
libraries providing an Application 
Programming Interface (API) for services 
such as indexing, search and relevance 
feedback. Stemming functions are provided 
for English and many other European 
languages. The library is written in C++, but 
APIs to other languages such as Perl and 
Python are also available. The model 
provided as part of the search service is the 
Robertson/Sparck Jones probabilistic model 
(Robertson and Sparck Jones, 1976) and the 
weighting function used is BM25 (Robertson 
et al., 1995). Boolean and phrase search 
facilities are also provided. Examples of how 
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to use the search and index facilities are 
provided. 


Swish++ 
(http:/homepage.mac.com/pauljlucas/ 
software/swish/) 

Swisht++ (Simple Web Indexing System for 
Humans) is a Unix based indexing and search 
engine that has been ported to MS 
Windows. It is a C++ rewrite of Swish-E 
(http://sunsite.berkeley.edu/Swish-E/), a 
system developed for Web search. Swisht+ 
offers command line services for Web 
crawling, indexing and facilities to create a 
search server. Non-text documents such as 
Microsoft Word documents can be indexed. 
Swish++ at present only handles the English 
language. The search model used in the 
software is Boolean, but some form of ranking 
function is provided (the term weighting 
model is not specified). A significant problem 
with both systems is that neither Swish-E nor 
Swish++ has the ability to merge intermediate 
results: the user is expected to increase the 
memory capacity of the hardware in order to 
handle large collections. This is not an 
appropriate strategy for an information 
retrieval system. 


Senga (http://www.senga.org/) 

Senga (like Xapian) is a library of C++ 
routines for developing information retrieval 
systems. Unlike Swish++, the developer must 
write his/her own query and document 
parsers. No stemming facilities are provided 
(unlike Xapian). Senga therefore provides a 
basic library for IR systems development. 
Senga shares some programming effort with 
ht://Dig, a good example of collaboration 
between two development groups. The 
system appears to allow programmers to 
define their own weighting function, so in 
theory any term weighting model could be 
supported. Senga has the ability to support 
updates, therefore dynamic indexes can be 
supported. The author had problems in 
building executables from source code for this 
system: no answer was ever provided to his 


` query and development work appears to have 


stalled. 


ht://Dig (http://www.htdig.org/ 

The ht://Dig system has already been 
mentioned in connection with Senga. This 
system provides programs to-crawl Web sites 
(or Dig them in their nomenclature), merge 
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data from existing database or newly found 
documents, and finally to search document 
and word databases. Search facilities include 
ranking and Boolean models, as well as 
special functions for multiple keyword search 
and building synonym databases. The ranking 
function is an ad hoc one and scores words 
nearer the top of the document higher than 
those nearer the bottom. The ability to define 
a front-end Web page for search is provided. 
The system appears to be quite widely used 
and there is much development work being 
done on it. There appear to be some problems 
on Linux versions: the author wasted the best 
part of a day trying to fix an intermittent bug 
that crashed the system and was impossible to 
find. 


Isearch (www.etymon.com/Isearch) 
Isearch was written to resolve perceived 
problems with freeWAIS, in particular that 
the search engine and retrieval protocol were 
mixed in together (Nassar, 1997). Apart from 
the article by Nassar (1997) there is very little 
documentation on the detailed facilities, but 
looking at the source code modules it is clear 
that functions such a geographical and date 
search are supported. Ranking is supported, 
but the model and functions used by the 
system are not declared. Nested Boolean 
searches are allowed, as is phrase searching 
and right truncation. Isearch gives the user 
the ability to index various formats such as 
HTML, SGML, Medline and USMARC. 
The code is no longer being developed and a 
replacement called Amberfish( (which is able 
to deal with structured XML documents) has 
been developed to replace it. 


Oasis (http://oasis-europe.org) 

Oasis (Kluev, 2000) differs from the systems 
described above, as it is a program for 
distributed search on the Internet. The base 
system used for search is Isearch (see above). 
It filters out collections using server selection 
methods and submits queries to distributed 
sites, merging the results from them. Oasis 
uses artificial intelligence and neural network 
techniques for server selection and query 
processing. The architecture used is basically 
three-tier middleware: the client sends a 
query to any Oasis server which contacts 
other Oasis servers available to obtain results. 
Oasis provides synchronous and 
asynchronous modes: in the latter the server 
acts as an intelligent agent. A collection 
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broker chooses the servers, but it is not clear 
how server selection is done. Results are 
merged using a neural network technique that 
also removes duplicate entries: it is entirely 
possible that duplicate sets of results can be 
generated in this type of architecture. Crawler 
services are provided for collection building: 
these are described in Kluev (2000). The 
system also supports relevance feedback. 


MG (http://www.cs.mu.oz.au/mg/) 
Managing Gigabytes or MG for short (Witten 
et al., 1999) is a text retrieval system that has 
been developed to investigate various aspects 
of compression for inverted lists. The code is 
released under the GPL, but is no longer 
being actively developed: further 
development is in the closed source model. 
The system provides facilities for indexing 
collections and searching them using either 
the Boolean or Vector Space models. The 
book by Witten et al. (1999) gives detailed 
information on the compression methods 
used in MG: there is also a guide to MG in 
the appendices of the book. 


Lemur 
(http://www-2.cs.cmu.edu/~lemur) 
Lemur is a toolkit which is being actively 
developed by both the University of 
Massachusetts and Carnegie Mellon | 
University to be used for research into the 
areas of language modelling and information 
retrieval. The toolkit allows programmers to 
develop all kinds of information retrieval 
systems such as cross-lingual IR and text 
filtering/classification. The system can either 
work on Unix or Windows systems. The 
licence used is a private one. 


The argument for open source IR 
systems 


A number of factors demonstrate that open 
source software development works well 
(Moody, 2001). The biggest factor is parallel 
development and debugging. In closed 
systems development, programming effort is 
restricted to a small team, the only people 
who can address problems in the code. It is 
difficult for team members to test their 
software against real problems. With open 
source software we may have programming 
effort shared among programmers who may 
be widely distributed all over the world. The 
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number of programmers who can tackle 
development is far more than any 
organisation that uses closed source 
development could ever deploy. Having this 
wide resource of programmers allows open 
source developers to distribute the functions 
of debugging and testing, deploying systems 
to real problems and identifying problems and 
errors which can be tackled far quicker than in 
closed source development. It is estimated 
that up to 70-80 per cent of the cost of 
software is on maintenance (Feller and 
Fitzgerald, 2002, p. 147): the potential 
economic benefits of sharing these overheads 
are clear. Another clear advantage is in the 
management of the projects. Although there 
will be one or two project leaders in open 
source projects, in the main each developer 
will work independently without the need for 
any significant direction (they volunteer to 
tackle a well defined specific problem) or the 
need for communication with everyone on the 
team. In closed source development, 
significant project management is required 

- which results in problems outlined by Brooks 
(1982), notably the offset between using more 
programmers on a project and the level of 
communication between them. 

The significance of this type of 
development is that new IR algorithms and 
models can be released into the community 
very quickly for practical application and 
deployment. This contrasts with IR ideas ` 
developed in closed source systems. These 
remain secret and unused, often untested and 
uncompared using standard test collections 
such as TREC (Voorhees and Harman, 
1999). These ideas are lost to the community 
who have no chance to discuss their impact. 
Porter and Boulton (2000) assert that mutual 
suspicion between IR practitioners in industry 
and IR researchers in academia was caused by 
closed and proprietary software development 
methods. To the industrialists, IR researchers 
did not understand the realities of dealing 
with real world problems and were stuck in 
theoretical ideas that do not tackle them. To 
the academics, people in industry did not 
understand the theory of IR and stole their 
ideas without giving them the credit they 
thought was due. Porter and Boulton (2000) 
argue that this lack of co-operation between 
the two camps has been damaging to both. 

While the survey in the section on “systems 
currently available” is by no means 
exhaustive, it is clear that some of the 
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arguments given here are illustrated by 
examples in open source systems. There is - 
clear co-operation. between Senga and 


ht://Dig as well as between Oasis and Isearch, > 


for the benefit of both sets of groups. 
However, it is also clear that some open 
source developers are completely unaware of 
progress made in IR theory: term-weighting 
functions used in many surveyed systems are 
either undeclared or. ad hoc, with little or no 
theoretical underpinning to them. It is very 
hard for software developers, who are 
interested in developing systems, to keep up 
with new ideas emanating from the world of 
IR research. ; 


Potential problems and obstacles 


Although open source development has 
shown in many cases to be successful, there 
are potential problems (Moody, 2001). 
Commercial companies do not always see the 
benefits, and are under significant pressure 
from shareholders who do not understand the 
open source development model. A case in 
point is BrightStation and Xapian. This (and 
other human factors) may cause forks. A’ fork 
occurs in open source development when a 
group developing open source software splits 
into two or more groups. This means that the 
groups will often duplicate work, wasting time 
and effort obviating the benefits listed above 
for open source development. 

The impact on IR system developers can be 
considerable. The author found 36 open 
source IR systems in the course of his - 
investigation. It is entirely possible that there 
are more systems available on the Web and 
from other sources. Proliferation may be no 
bad thing to begin with in order to share - 
ideas, but at some stage it would be a good 
idea to coalesce the ideas from these systems 
in order to get the best of them and hence to 
share them. In the long run it is desirable that 
only a few systeris be available for redsons 
given above. However, the author does not ` 


` want to put off IR researchers or developers 


from releasing their code: going’ open source 
is another way of sharing ideas. One of the 
main reasons-for publishing the material here 
is to encourage IR practitioners and 
researchers to share ideas, coalesce them from 
different systems and to prevent forks before 
they happen. The author accepts that it will 
be impossible to coalesce all the open source 
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IR systems into one for significant: political, 
social and software engineering reasons (e.g. 
differences in programming languages). 
_ There is potential for wasted effort in 


“finding problems in the software. For 


example, we may have many programmers 
looking at the same chunk of code, 
duplicating debugging effort and therefore 
reducing overall efficiency of the open source 
software development model (McConnell, 
1999). There is also a workload deployment 
issue: for example some programmers have 
stronger expertise in some areas than other 
programmers (McConnell, 1999). There is an 
offset when maintaining software in the 


` benefits of sharing maintenance and loss due 


to wasted effort. This is a significant area of 
research in open source software (Feller and 
Fitzgerald, 2002). 

Another significant factor and one that has 
received little attention is usability. Nichols et 
al. (2001) argue that open source software 
projects need to adapt in order to produce 
systems that can be used by a typical and 
non-technical user. The basic problem is that 
most open source systems are written by 
programmers who do not understand end 
user needs and whose software is often 
complex and difficult to use. For example, a 
quick look-at some of the indexing methods 
used by systems in this survey has revealed 
how difficult and complicated creating an 


‘index is., Often. significant technical 


knowledge is needed in order to make the 
process work correctly or efficiently. This is a 
serious problem, and given the importance of 
interaction in IR systems it is one that must be 
addressed by open source IR developers. 


Conclusion: the way forward 


The benefits of sharing ideas between IR 
industrialists and researchers are clear. People 
in industry often have access to users which 
IR researchers can only dream of. Ideas can 
be tested out in real situations, often 
anonymously without reference to the users 
or publishing the material. These ideas can be 
embodied. in the systems themselves and 
downloaded for use in either research or 
deployment in real world search. This author 
believes that good new ideas should be 
available to all, and both IR industrialists and 
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academics stand to gain from closer 
co-operation. Programmers who work on 
open source IR systems should be encouraged 
to consider the issue of usability and where 
possible to coalesce their systems. The author 
is aware, however, that efforts are being 
dissipated in the development of open source 
IR systems, and may thus prevent the obvious 
benefits in using the open source 
development model. It is hoped that further 
research into the open source development 
model will be able to show tangible benefits, 
and that this will impact positively on using 
such a model to produce IR systems. 
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Appendix 


Table Al Other open source/free software IR projects 


Project name 


URL 
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Alkaline 
Amberfish® 
Glimpse 
Ongobongo 
Perlfect 
ROADS 
SEARCH.PHP3 
YASE 

Exist 
PhpDig 
NISs 
Anarchivist 
ASPSeek 
HISS 
Latente 
NeatSeeker 
Nose 
OpenFTS 
Oxyus 
Ransacker 
siteIndexer 
SPINdex 
Doc Fox 


MPS Information Server 


Lucene 
freeWAIS-sf 
Harvest 
Zebra 


Personal Librarian 


alkaline.vestris.com 
www.etymon.com/amberfish 
webglimpse.net 

ports.tolkien.dk/ongobongo 
perlfect.com/freescripts/search/ 
www.roads.lut.ac.uk 
www.w3webmaster.com/search/install.shtml 
www.mazumdar.demon.co.uk/yase_index.html 
exist.sourceforge.net 

phpdig.toiletoine.net 
sourceforge.net/projects/niss 
sourceforge.net/projects/anarchivist 
www.aspseek.org 
sourceforge.net/projects/hiss 
sourceforge.net/projects/latente/ 
neatseeker.sourceforge.net 
sourceforge.net/projects/nose 
openfts.sourceforge.net 
sourceforge.net/projects/oxyus 
ransacker.sourceforge.net 
sourceforge.net/projects/siteindexer 
mattwork.potsdam.edu/projects/spandex 
sourceforge.net/projects/docfox 
www.fsconsult.com 
jakarta.apache.org/lucene/ 
Is6-www.informatik.uni-dortmund.de/Ir/projects/freeWAlS-sf 
www.tardis.ed.ac.uk/harvest 
Indexdata.dk/zebra 

www.pls.com 
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Stage Licence Other information 
Stable Private Free for non-commercial use 
Stable GPL Process structured XML 
Stable Private Free for non-profit organisations 
Alpha GPL Uses PostgresQL 
Stable GPL - 

Stable GPL Yahoo like 

Stable GPL Uses MySQL 

Beta GPL - 

Beta LGPL Uses RDMS 

Stable GPL Uses MySQL 

Planning GPL - 

Planning GPL Rewrite of AustLll 
Stable GPL Uses SQL 

Planning GPL - 

- - Uses Java 

Stable Apache - 

Planning GPL - 

Stable GPL Uses PostgresQL 
Pre-Alpha Apache - 

Alpha GPL Uses Python 

Alpha GPL Uses MySQL 

Stable GPL - 

Planning BSD - 

Stable Private - 

Stable Apache Uses Java 

Stable Private - 

Stable Private - 

Stable Private’ - 

Stable Private ~ 
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Introduction 


This article reports on the utilisation of an 

integrated training resource programme 

consisting of a mix of media and technologies 

for the education and training of cataloguing 

students of Technikon SA, a distance 

education institution in South Africa. 

The mix included: 

e printed study texts; 

* aCD-ROM-based computer program 
composed in Authorware; 

+ Internet-based virtual campus facilities; 

* communication channels such as e-mail; 
and 

* contact classes. 


The training programme was used for training 
cataloguing students in contact classes, 
distance education and in-service training. 

The purpose of the research on which this 
article reports was two-fold: first, to 
determine the advantages and disadvantages 
of training cataloguers by means of such a miz 
of media and technologies; and second to 
establish design criteria for mixed-modal 
training of cataloguers. 


Background 


The Department of Library and Informatior. 

Studies at Technikon SA offers training in 

library and information science through 

distance learning to students who already 

work in libraries and information services as 

administrative assistants and who wish to 

become fully qualified librarians and 

information workers. Students work in a wide 

range of environments: 

* community libraries in rural areas; 

* public libraries in urban areas; 

* academic libraries at universities and 
technikons; and 

* specialised information centres. 


The core modules of the library and 

information studies curriculum are: 

* library and information practice; 

* information retrieval (including 
cataloguing training); 

°- library and information technology; and. 

* user studies. 
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The training of cataloguers forms an 
important part of the education and training 
of librarians since cataloguing is still 
considered one of the most important skills 
for organising information (Smith, 1985; 
Stieg, 1992; Clack, 1993). Bender (cited in 
Bearman, 1987) mentions that: 


... traditional skills associated with acquiring, 
organizing, and disseminating information will 
still be needed by tomorrow’s information 
professional at the entry level. 


One of the most important skills for 
organising information is cataloguing. The 
automation of libraries has also changed the 
nature of the work in cataloguing 
departments. The functions and tasks of 
librarians in cataloguing departments have 
changed. However, bibliographic work can 
never be completely computerised. Human 
input, especially with regard to authority 
control, is very important (Snyman, 1998). 
Co-operative cataloguing has become an 
“important function amongst libraries. Copy 
cataloguing is now one of the most important 
tasks of cataloguers. To adapt and utilise a 
copy record for a library’s own purposes, it is 
essential that the cataloguer have a sound 
knowledge of cataloguing principles. Such 
knowledge can only be achieved through 
proper education and training in cataloguing. 
Zyroff (1996) mentions a number of valid 
reasons why training in cataloguing should 
now, even in the age of automation, 
co-operative cataloguing and shared 
cataloguing, still be considered a core 
competency. Her statement that: 
... [t]hose who have not spent time applying and 
creating subject and name headings, authority 
records, descriptive cataloguing and 
classification codes, and indexing norms don’t 


have in-depth perspective on the structure of 
information ... 


captures the essence of organising 
information. Cataloguing skills also play an 
important role in the organisational aspects of 
knowledge management (Hill and Intner, 
1999). It has therefore become more 
important than ever that cataloguers receive 
appropriate education and training through 
all possible modes. 

Cataloguing training is usually conducted 
by the following means: 
* contact training; 
+ distance learning; and 
* in-service training. 
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There are some limitations associated with 
the above-mentioned training means. 

During contact training students attend 
lectures and conduct practical sessions where 
students practice cataloguing techniques. At 
the contact training institutions it is not 
always possible to expose students to 
sufficient practical training. Lecturers’ time is 
restricted to the lecturing periods and student 
assistants are usually not experienced enough 
to facilitate all the required practical training 
sessions effectively. 

The education and training of Library and 
Information Science students in South Africa 
through distance learning is becoming more 
popular, since an increasing number of 
students are unable to afford the luxury of 
full-time studies at a contact class institution. 
Distance learning has traditionally been 
provided at institutions such as Technikon SA 
through paper-based correspondence. In 
distance education as a method of tuition the 
written word is the main medium of 
instruction. Interaction between the lecturer 
and student is limited to course material and 
books, completed assignments from the 
students and projects related to their work 
experience. This low level of interaction 
between lecturer and student has led to a 
number of problems in the education and 
training of cataloguers. Students often need 
an immediate answer to a question or 
problem in a practical exercise for them to 
continue to the next step. With paper-based 
distance education they usually have to wait 
weeks to receive an answer to problems 
encountered during the study of a particular 
skill. Other delivery modes such as audio, 
video and computer technologies are also 
utilised in distance education (Steiner, 1998), 
but the time delay in interaction between the 
student and lecturer still exists. Asynchronous 
instruction has the advantage that students 
may choose their own instructional time, but 
there is no real-time interaction and 
immediate feedback from the lecturer. 

Usually, when graduates enter cataloguing 
posts they need a refresher course in 
cataloguing conducted by the supervisor or 
senior staff member of the department. They 
also need in-service training to understand 
and master the specific procedures at the 
library where they are employed. Library 
assistants or clerks (without any formal library 
education or training) are often employed in 
cataloguing departments to perform certain 
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routine, administrative cataloguing tasks. 
They are given in-service training by the 
supervisor or senior staff member. These 
in-service training periods take about six to 
nine months to complete. During that period 
the trainee cataloguer is not very productive 
as far as work output is concerned. Moreover, 
the supervisor or senior staff member 
conducting the training has to perform this 
function, which includes evaluating the 
trainee’s work and discussing problems, on a 
full-time basis. This means that this senior 
staff member does not perform other 
cataloguing tasks for which he/she has 
originally been appointed. 

Henderson (1987) suggests that 
cataloguing students should be presented 
with sound educational experiences and that 
enthusiasm for the subject should be fostered. 
Hill (1985) states that too few students are 
exposed to the fun of cataloguing and do not 
appreciate the intellectual exercise. Real 
meaningful learning does not take place in the 
classroom setting, but only when students 
have the opportunity to apply the instruction. 
Students should have the opportunity to work 
on real problems under real conditions 
(Zuber-Skerritt, 1993). Such opportunities 
are referred to as active learning, where the 
students approach the course content through 
problem-solving exercises (Romero, 1995). 
Apart from problem-solving skills, analytical 
thinking and decision-making skills form an 
important part of cataloguing (Connaway, 
1997; Olson, 1997). De Boer et al. (2001) 

_ _Teport that the thinking preferences of 
cataloguing students at the University of 
Pretoria do not correspond to the skills and 
techniques required in cataloguing. The 
students specifically do not like analysing, 
problem solving, implementing and 
organising skills. Yet, these skills are essential 
for cataloguing tasks. The suggestion is that 
more attention should be given to the less 
preferred but essential preferences; and that 
teaching methods should be adapted to use 
preferences to develop needed skills. 
Activities such as simulations, case studies 
and small group activities could all form part 
of active learning. This is especially applicable 
to a cataloguing course. 

A training resource that includes interactive 
instruction with the utilisation of multimedia 
could provide this meaningful learning 
opportunity. 
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According to Clack (1993), cataloguing 
teachers and trainers: 


... Must be innovative in their teaching strategies 
and methodologies. 


Training should not only be done for present 
practice, but also for the future. 


The right tools for the task 


It has been said that “ ... if your only tool is a 
hammer, every problem looks like a nail... ” 
Multimedia provides us with more tools. It is, 
however, important to choose the right tool for 
the task (Folkers, 1994). 


According to the well known and much 
disputed “no significant difference 
phenomenon”, no matter who or what is 
being taught, the utilisation of various media 
will produce the same learning results. 
Evidence for this argument is based on 
hundreds of media comparison studies 
produced since the 1920s (Russell, 1999). 
Russell, however, does believe: 

... that differences in outcomes can be made 


more positive by adapting the content to the 
technology. 


In going through the process of redesigning a 
course to adapt the content to the technology, 
it can be improved. 

The question with regard to the training of 
cataloguing students is: would these students 
not learn from traditional print-based 
correspondence courses as well as they would 
from various other media and technologies? 
To answer this question, a study similar to the 
many quoted by Russell (1999) would have 
had to be conducted, specifically focusing on 
cataloguing students. The result would 
probably have been the same: no significant 
difference. The purpose of this study, 
however, is not to find a few technologies or 
media that might achieve improved learning 
results, but to establish how the available 
technologies and media should be utilised and 
combined to provide an active learning 
environment for cataloguing students. 

The utilisation of educational technology, 
especially computer-aided and Web-based 
training, influence education design and 
development tremendously. Outcomes-based 
education and educational technology play a 
vital role in the design and development of 
training and have to be considered in such 
studies. 
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We needed to determine whether 
developments have taken place in the 
utilisation of a mix of appropriate media and 
technologies in the training of cataloguers, 
and if so, what these developments are, in 
order to adapt existing developments or 
develop new training resources. The question 
was investigated by means of an analysis of 
reported research and by evaluating a training 
resource consisting of a mix of media and 
technologies. The training resource consisted 
of: 

* print material (study guides, tutorial 
letters and prescribed books); 

* practical exercises (drill exercises in print 
format); 

* contact classes; 

* communication channels, such as e-mail; 

* a mentor system; 

* a Web-based system (Technikon SA 
Co-operative Online Learning (TSA 
COOL) virtual campus); and 

* a computer-based tutorial and simulation 
program. 


For the purposes of this study it is believed 
that the study guide that consists of printed 
study text as well as drill exercises should 
form an integral part of a training resource. 
All the students receive study guides and 
printed material is considered the main 
medium for training. 

Contact classes involved group discussion 
opportunities conducted monthly in the 
regions where the largest student numbers are 
concentrated. 

Communication channels included 
telephone, fax and electronic mail. 

The mentor system is a form of experiential 
learning conducted at libraries where students 
do practical projects under the guidance of a 
mentor. Students are advised (and 
encouraged) by duly qualified and 
experienced practitioners (or mentors). 

TSA COOL is combined with the 
traditional print-based distance education 
media and could therefore be referred to as a 
hybrid campus. TSA COOL partially 
supports a constructivist paradigm in that it 
encourages collaborative learning. 
Collaboration takes the form of group 
activities, a discussion group facility, links to 
sites with related topics, an e-mail facility to 
the authors of the site and contact with live 
subject matter experts. TSA COOL promotes 
discovery learning in that it has links to other 
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organisations’ Web sites. The welcome screen 
of TSA COOL is shown in Figure 1. 

In Figure 2 the discussion group facility is 
illustrated. 

The computer program provides additional 
practice for cataloguing students and 
includes: true and false questions, 
multiple-choice questions, completing 
diagrams and flow charts, and placing books 
in their correct positions on shelves. Figure 3 
is a screen shot of a book classification 
exercise of the computer program. 


Research methods 


The training resource was developed as a 
combination of various media and 
technologies to present the cataloguing 
course. The research, conducted to determine 
the implications of such a multi-modal 
approach, was primarily qualitative. 
Questionnaires, interviews, observations and 
focus group discussions were used to evaluate 
students’ experience of the resource. A 
pre-test/post-test design, or a media 
comparison study, was not conducted, as we 
accept Russell’s (1999) proposal that the 
choice of medium would make no significant 
difference to the academic results. 
Questionnaires were sent to all the students 
(109) registered for the first registration 
period of 2001. Questionnaires were used to 
test the experience of the wider population of 
students. The purpose of the questions was to 
obtain simple reflections from the student’s 
recent experience with the programme. 
Students completed the questionnaire after 
spending some time using the resource. 
Observations were made and interviews and 
focus group discussions were held with 
students (individually and in groups of five to 
ten students) who could attend contact 
sessions at Technikon SA’s main campus in 
Florida (Johannesburg, South Africa). 
Observations entailed that the researcher 
was present with some of the students while 
they worked with the components of the 
training resource, to record noteworthy 
incidents and comments. Laurillard (1994) 
considers observation an extremely rich and 
useful data collection method that can be 
carried out very quickly. The unconstrained 
data collection process complements the 
closed evaluation techniques such as 
questionnaires. In the evaluation exercise one 
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Figure 1 Welcome screen of TSA COOL 
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Figure 3 Book classification exercise 
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not only where the resource fails, but why, 
and what needs to be done about it. One 
must, however, keep in mind that what 
students say they “like” or “prefer” may not 
necessarily be the best for learning. 

The open-ended nature of interviews can 
help to establish what aspects should be 
further investigated with the closed form of 
the questionnaire (Laurillard, 1994). On the 
other hand, certain information obtained 
from questionnaires can be further 
investigated in greater depth by interviewing 
respondents to the questionnaires. Interviews 
and informal discussions were held with the 
students who used the resource. These 
interviews were conducted after the 
questionnaire had been completed. The 
interviews and comments provided the 
opportunity to clarify certain comments made 
by the students and aspects indicated in the 
questionnaire. 


Findings 


The aim of this research was to determine the 
implications of a blended approach to the 
training of cataloguers. Initial findings from 
the literature and evaluation of the training 


resource programme are summarised as 

follows (Cloete, 2001): 

* The utilisation of various media will 
produce the same learning results. 

¢ Library and information study students 
often receive lectures or notes about 
computers and computer software rather 
than hands-on experience or 
computer-aided simulations. 

* In the international literature few 
references are made to the utilisation of 
computer-aided instruction in 
cataloguing training. 

* No references in the national literature 
could be found to the use of 
computer-aided training in cataloguing 
courses. 

* Drills, practice and simulations are the 
instructional strategies that could be 
considered for a mix of media and 
technologies. 

* Students were generally satisfied with the 
training resource programme. 


A comparison between suggestions from the 
reported research regarding online training in 
general and findings of our evaluation of the 
blended training resource programme for 
cataloguers is shown in Table I. 
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Table 1 Comparison between suggestions in literature and findings from evaluation 


Literature suggests 


Findings from evaluation 





> Students expect lecturers to respond to their e-mail messages 
almost immediately and can be more demanding online than in 
person (Rea et al., 2000) 


Lecturers must spend time and energy on the virtual campus site 
to ensure that the site is working properly and all information is 
up to date. This means extra work, but this tends to decrease 
over time (Rea et al., 2000) 


Students expect information such as the syllabi, previous 
examination papers and notices on the virtual campus site 
(Rea et al., 2000) 


The utilisation of drills and practice helps students to review, 
remediate, rehearse and practice (Erickson and Vonk, 1994) 

Immediate feedback and remediation to students’ responses is very 
important (Hannafin and Peck, 1988) 


Competition is considered an effective motivating technique 
(Erickson and Vonk, 1994) 


From the above comparison it is evident that 
findings from the evaluation correspond 
mostly with suggestions from the reported 
research. We then needed to determine the 
advantages and disadvantages of utilising a 
mix of media and technologies in order to 
develop training that best suits the training 
needs of cataloguing students. The findings 
from the reported research and evaluation of 
the training resource programme to answer 
this question are summarised as follows 
(Cloete, 2001). 


Advantages 

* Students find it more interesting to work 
with different media and technologies. 
The limitations of one medium such as 
print can be overcome by the variety of 
other media and technologies. 

+ Students can work on the course 
whenever it suits them. 

* Neither the distance that geographically 
separates students nor the time of day 
when it suits them to study placed 
limitations on their progress. 

>+ + A degree of individualised instruction was 
possible. 

* The different media and technologies 
could provide a variety of information 
and additional exercises. 


Students send a follow-up e-mail within a couple of hours to a day if a 
response to an original e-mail was not sent. Even if they did not ask a 
question, but only e-mailed work, they want a response. The same, 
however, also applies to telephone messages and faxes 

Initially it took time to set up all the information, links, FAQs, discussion 
groups and academic guidance for the subject on TSA COOL. Once the 
site has been set up, it has to be monitored regularly for students’ 
inputs. Outdated information is removed once a month end new 
information is added as it becomes available 

Once students became aware that previous examination papers were 
available on TSA COOL, they expected them shortly after each 
examination period and made enquiries if the papers were not 
immediately available 

Students found the drill-type exercises in the computer program and 
questions for self-assessment on TSA COOL very helpful for practice 

Students expressed their appreciation for the immediate feedback and 
explanations in the computer program and TSA COOL's questions far 
self-assessment 

Students found it challenging to have their performance assessed in the 
computer program and TSA COOL's questions for self-assessment and 
to have the opportunity to improve on their scores 


* The electronic media made it possible to 
train students in the specific skills of the 
cataloguing course, which was not 
possible through the print media. 

+ The electronic media made it possible to 
create simulations of cataloguing 
situations. 

e Immediate feedback was possible through 
the facilities on the virtual campus and 
the computer program. 

+ Testing and retesting was possible with a 
computer. 

* The variety of computer-based training 
stimulated students and promoted 
positive attitudes to learning. 

* It was possible for the student to navigate 
the content. 

* Connectivity enabled belonging to a 
discussion group so that students had the 
opportunity to give and share their 
opinions. 

* The computer program and TSA COOL 
encouraged group work and collaborative 
learning, which helped students to get to 
know each other better, and reduced the 
social isolation associated with distance 
learning. 

e Students could reflect and learn from 
other students when they participated in 
the discussion group facility on TSA 
COOL. 
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Disadvantages 

* The electronic media were still very new 
to the students and they were therefore 
reluctant to use them. 

+ Students had difficulty in obtaining 
access to the Internet. 

* Students did not all have adequate 
hardware or appropriate software to 
access the multimedia and interactive 
capabilities of the virtual campus. 

- It took time and technical support to 
become a competent user of the virtual 
campus. 

+ Students considered the use of the 
electronic media and technology as extra 
work in an already demanding course. 

* Lack of computer skills created problems 
for certain students to adjust to the 
electronic media. 


We needed to determine how training by 
means of a mix of media and technologies 
should be designed in order to develop an 
appropriate training resource. To achieve this, 
it is necessary first to summarise what we had 
learned from this study. Second, guidelines for 
the design and development of a mixed mode 
training resource will be formulated. Third, 
possible future developments will be 
investigated. These issues will be discussed in 
the following sections. 


What we learned from the study 


+ Students use media and technologies to 
meet their needs and not necessarily in 
ways expected by the lecturer. 

* Students with little understanding of or 
familiarity with computers will find the 
use of media and technology difficult at 
first, and will have to spend more time on 
the media and technology than on the 
contents of the course. 

* Frequent communication with students is 
essential. 

+ No assumptions should be made about 
students’ abilities, experience, etc. A 
student profile should be compiled first. 

+ Students should be told exactly what they 
can and should do with each component 
of the training resource programme, for 
example what each function of a virtual 
campus is for and how to use it. 

« Merely placing course material on the 
virtual campus, i.e. using it as just 
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another delivery mode, should be 
avoided. 

e Students do not use all their study 
material, but only what they consider 
essential for passing the course. 

+ The utilisation of a mix of media and 
technologies allows interaction between 
students, lecturers and sources of 
information. 

* Many frustrations can be related to 
technical problems, e.g. Internet 
connections and download time. 
Technical problems diminish as students 
progress with the course. 

+ Students tend to forget that there is a 
variety of communication channels to the 
lecturer and fellow students. They often 
complain that geographical distance 
makes it impossible to benefit from the 
training. They need to be reminded of all 
the possible channels such as e-mail, fax 
and discussion groups. 

e There is no definite preference for using 
the computer program in a group or 
individually. Students who worked on it 
during observations preferred to work in 
groups. 

+ The immediate feedback provided by the 
programme, which includes correct 
answers as well as explanations, was 
highly appreciated. 

+ Students start to use one component of 
the programme, such as the group 
discussion facility on the Internet, to 
organise another component, namely 
additional contact classes amongst 
themselves. 


Guidelines for designing and developing 
a training resource programme 


From the analysis of the literature and the 
evaluation of the training resource 
programme, it is now possible to derive the 
following guidelines for the design and 
development of a training resource 
programme consisting of a mix of media and 
technologies. 

°- A user-friendly design is essential to make 
all the components of the programme 
accessible to all students. 

* Frustrations and problems related to the 
technical aspect should be counteracted, 
as such problems affect how students 
perceive and experience the programme. 
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Support should be available for all 
students. Networks must be stable to 
avoic problems with access and 
passwords, problems accessing 
information, slow response times and 
unre‘iable features. 

+ The programme should be suitable for 
group work or individual work. 

* Immediate feedback in certain 
components of the programme is 
essential. Students need to know the 
correct answers immediately and, where 
applicable, explanations of incorrect 
answers should be given. 

+ Performance measurement is helpful to 
the students. This could be done in the 
same way as the score ratings given in 
many computer games. 

¢ Navigation must be satisfactory. Students 
should be encouraged to read the 
guidelines and instructions before 
attempting the programme. Navigational 
help files should be made available at all 
times. 

* There should be frequent, active 
involvement from the learner and 
lecturer, who, in fact should act more as a 
facilitator than a lecturer. 

* Hypertext linkages should be used. These 
could be electronic links between related 
information chunks to expand ideas, 
learn related concepts, view document 
sources and stimulate additional learning. 
Hypertext linkages could provide 
immediate access to the most recent 
information. 

+ Multimedia (sound, graphics, colour and 
moving images) should be used 
functionally to enhance learning. 

* Visual appeal: there should be a polished 
look to the programme with the colour, 
graphics, animation, etc. 


* Communication channels such as e-mail 


and discussion groups should provide 
options for frequent interaction. 

* Records of students’ participation should 
be kept. 

+ Jt should be easy to update all 
information. 

¢ Alternative instructional strategies should 
be created, each with appropriate 
multimedia materials, and students 
should be directed toward that strategy 
found to be most effective for their style. 

+ Individual students should be encouraged 
to participate in the discussion group 
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facility. As in the contact classroom 
situation, students often do not respond 
to a question or statement if it is posed to 
them in general, but have to respond 
when they are addressed directly. In the 
discussion group, students could be 
encouraged by rewarding them for 
responding; for example five marks could 
be added to an assignment. 

+ Students working alone as well as those 
working in groups must benefit from the 
media and technologies. 

* Regardless of the technical support 
provided, the lecturer must have a good 
knowledge of the media and technologies 
and be prepared to answer questions on 
equipment as well as content. A key 
feature should be that a variety of 
methods of interaction are available. 

* A smaller pilot project that focuses on 
specific learners should be implemented. 
A traditional course should be run 
parallel to the paper-based system. 

¢ It is essential that the lecturer gives 
students the necessary guidance and 
motivation they need for participating in 
the training programme. 

-+ A great deal of interaction with students is 
necessary for them to comprehend the 
course material and new media and 
technologies effectively. 

* The mixed mode training resource should 
function as a coherent whole. 

+ A training resource that suits the training 
institution’s budget should be considered. 


The possibilities of a training resource 
programme consisting of a mix of media and 
technologies for cataloguing students and 
their implications for instructional design and 
promotion of learning could be summarised 
as shown in Table II. 


Developments towards a virtual 
cataloguing training classroom 


The ultimate goal of the training resource 
programme was to develop an online, fully 
interactive course: an online virtual 
cataloguing classroom. This means that 
technology will be incorporated as a 
substitute for the classroom and traditional 
distance training methods. The students 
may never meet the lecturer in person. The 
virtual cataloguing classroom should 
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Table 11 Possibilities and implications of a training resource programme consisting of a mix cf media and technologies 


Printed course Communication 


Web-based 
training and 





material Contact classes channels Mentor system Computer program communication 
Uses Provides the contents Opportunity for E-mail, fax, Part of the Incorporation of Incorporation of 
of the course face-to-face telephone and experiential multimedia to multimedia to 
material and contact with the discussion group training provide an provide an 
tutorial material. lecturer and fellow facilities can be component. element of reality. element of reality. 
Includes exercises students. Students used to Students conduct Graphics, sound Graphics, sound 
and activities to can discuss communicate with work-related and text attract and text attract 
be answered in content problems the lecturer and projects attention. attention. 
print format and clarify issues fellow students to (cataloguing) Incorporation Incorporation of 
clarify course under the of problem-solving problem-solving 
content problems supervision of situations situations. 
or just exchange a mentor Provides links to 
ideas and useful Web sites 
comments and documents 
Implications Must be designed Classes should Students must be Clear guidelines Students must Students must 


should be 
provided to the 


reminded of these 
opportunities and the 


very carefully with revolve around 
clear learning students’ needs and 


become involved 
in the learning 


become involved 
in the learning 


objectives to be 
met by the 
contents and 
evaluation 
methods applied 
in the training 
resource 


not become merely 
formal lectures 


responsible use of 
them should be 
explained to them 


students and 
mentors of what 
is expected from 
them 


process. They 
must be engaged 
in solving 
problems related 
to cataloguing 
issues. Contents 
must ba in 
audio-visual 
format with a high 
level of 
interactivity. 

Easy to use 


process. They 
must be engaged 
in solving 
problems related 
to cataloguing 
issues. Site should 
incorporate 
interactivity. Site 
must be engaging 
and stimulating. 
Useful. Organised 
and easy to use 











incorporate all the technologies already used 
and discussed. The course will then be 
offered over the Internet only. Weaknesses 
of this approach lie in the fact that students 
may become disassociated with the course 
since they do not feel involved. Lack of 
personal interaction may also contribute 

to this. 

Since students are familiar with the 
traditional classroom set-up, the classroom 
metaphor could be kept. The classroom 
would serve as the interface in which students 
could present their own work and view the 
work of their fellow students. They can thus 
enter into the spirit of a physical classroom. 
Training of librarians in a virtual 
environment is further motivated by the 
move towards the virtual library or “library 
without walls”. Soon more and more 
librarians will work in such an environment 
and therefore similar training would be 
considered appropriate. 


Concluding remarks 


Many of the requirements of trained 
cataloguers cannot be met in the traditional 
contact class or through the traditional 
distance training methods. The opportunities 
provided by the mixed mode training resource 
researched here should contribute to 
improved knowledge and interaction. It 
would offer students many more possibilities 
of interacting and being active in mastering 
the cataloguing course. 

A number of advantages of utilising a mix of 
media and technologies were identified and 
are summarised as follows: 

(1) Multimedia capabilities of the virtual 
campus and computer program mean 
that learners can access a variety of means 
of representing information. They can 
also be afforded opportunities to engage 
in active learning, working both alone and 
in groups. 
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(2) Students will not focus on the lecturer but 
rather on their own learning process. The 
tools provided by a training resource 
programme assist the students to rely on 
their own learning rather than being 
directed all the time. Lecturers are 
becoming learning facilitators. 

(3) The fact that an overwhelming majority 
of students would use the training 
resource again and recommend it to 
others is an indication that further 
multimedia developments of the course 
are necessary. 

(4) Apart from specific subject knowledge, 
the goal of the initial training of 
cataloguers must necessarily include basic 
training in the use and exploitation of 
media and technologies. The media and 
technologies play a threefold role: 

+ they are essential computing and 
communication tools; 

+ they are tools used in computerised 
cataloguing; and 

e they are a training resource, 
incorporating the most appropriate 
media designed to best meet the 
learning needs. 
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Introduction 


We are currently finding ourselves in the 
middle of the information age, suffering from 
information anxiety caused by “infoglut”. 
Research published by the management 
consulting firm KPMG in March 2000 found 
that 70 per cent of the companies surveyed 
claimed that they suffer from information 
overload despite implementing a variety of 
technologies such as corporate intranets and 
data warehouses. In spite of this apparent 
information overload, Gartner (Jacobs, 2002) 
claims that we are at the same time 
experiencing “infofamine”, in terms of access 
to knowledge. In the average US company, 80 
per cent of digitised information is stored in 
personal files on individual personal 
computers. Less than 5 per cent of employee 
knowledge is actually captured and accessible 
across the enterprise (Wells et al., 2000). 

More and more, enterprise portals (EPs) 
are being presented as the antidote to the 
infoglut and infofamine phenomena, and as 
the tool to revolutionise our access to 
information and knowledge. Portal 
technology has gripped the imagination of 
information technology (IT) users and 
vendors ever since the popularisation of the 
concept at the end of 1998 (Wells et al., 
2000). It is predicted that the growth of the 
portal market will be phenomenal. 

In 1999 Gartner (Phifer, 1999) stated that: 


... by 2001 the knowledge portal will become a 
standard component of successful knowledge 
management implementations. 


This prediction is supported by the Meta 
Group (Cain, 1999), claiming that by 2002: 
[K]nowledge management collaboration and 

innovation will permeate the Global 2000 
corporation’s strategy. However, more than 90 
per cent of the G2000 will recast knowledge 
management efforts as portals, delivering 
personalised, in-context information gathered 
through internal and external sources (Cain, 
1999). 


Ovum (Wells et al., 2000) estimated that the 
total market for portal software would be 
worth almost $1b in 2000 and would grow to 
more than $7b by 2005. This kind of hype 
about portal software tends to hide the key 
message of the knowledge management (KM) 
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movement that people, not technology, solve 
information and KM problems. 

The aim of the article is to provide an 
overview of the relationship between KM and 
IT and between KM and EPs. More 
specifically, it will be illustrated that 
implementing an EP is not implementing 
KM; an EP is only an IT tool in the 
implementation of KM. To test this theory, a 
case study is used to describe the practical 
experiences of an organisation where an EP 
has been implemented. The case study will 
determine to what degree the particular EP 
was implemented as part of a KM strategy 
and which KM elements were involved in the 
implementation. Through the case study best 
practices will be identified for the 
implementation of an EP. Information for the 
case study was gathered through interviewing 
the knowledge manager at an IT consultancy. 


The relationship between KM and 
technology 


In trying to define KM, O’Dell and Grayston 

(1998) say: 
When explicitly managed, organisational 
knowledge is used to accomplish the 
organisation’s mission. Knowledge management 
is therefore a conscious strategy of getting the 
right knowledge to the right people at the right 
time and helping people share and put 
information into action in ways that improve 
organisational performance ... it is a framework, 
a management mind-set ... (O’Dell and 
Grayston, 1998). 


However, the term “knowledge management” 
can be an unfortunate term since it promises 
the painless planning, organising, directing 
and controlling of knowledge (which is 
unstructured), in the same way as is done 
with structured items. Therefore, for many 
managers, the term promises a simple IT 
solution that treats knowledge in the same 
way as information or data. Actually, in order 
to obtain leverage on the knowledge within 
organisations, what is needed is far more 
subtle and widespread — it amounts to a whole 
new way of organising and thinking 
(Widmayer, 2000). 

The elements that need to form part of KM 
can be illustrated by means of MITRE’s 
model, which provides a holistic approach to 
KM - where KM is viewed from a 
two-dimensional perspective. The 
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components of the model were derived from 

KM research performed in MITRE’s 

technology program and a survey and 

comparison of KM models (e.g. Ernst & 

Young, APQC and DataWare) being used in 

industry or described in the literature (Taylor 

Small and Tatalias, 2000) (see Figure 1). 

The first dimension consists of the activities 
that are critical for knowledge creation and 
innovation: knowledge exchange; knowledge 
capture; knowledge re-use; and knowledge 
internalisation. Collectively, these processes 
build a learning organisation skilled at 
creating, acquiring and transferring 
knowledge as well as adapting its actions to 
reflect new insight and innovation (Taylor 
Small and Tatalias, 2000). 

The second dimension consists of those 
elements that enable or influence 
knowledge-creation activities. These include: 
+ Strategy — the alignment of corporate and 

KM strategies. 

* Measurement — the measures and metrics 
captured to determine if KM 
improvement is occurring or if a benefit is 
being derived. i . 

e Policy — the written policy or guidance 
that is provided by the organisation. 

* Content — the corporate knowledge base 
that is captured electronically. 

* Process — the processes that knowledge 
workers use to achieve the organisation’s 
mission and goals. 

* Technology — the IT that facilitates the 
identification, creation and diffusion of 
knowledge among organisational 
elements within and across enterprises, 
e.g. an EP. 


Figure 1 MITRE’s KM model 








Source: Taylor Small and Tatalias (2000) 
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* Culture — the environment and context in 
which KM processes must occur (Taylor 
Small and Tatalias, 2000). 


This model illustrates that technology is only 
one element of KM. In order to achieve 
successful and effective KM within the 
organisation, all the elements mentioned 
above should be present. KM is not the 
implementation of technology; rather, it is a 
multi-disciplined approach that integrates 
business strategy, cultural values and work 
processes. KM programmes perform best 
when enabled with sophisticated technology, 
but an emphasis on technology alone will 
achieve very little progress towards KM 
(Harris et al., 1999a). 

As early as 1997 Gartner (Bair et al., 1997) 
predicted that, as KM matures into a 
megatrend, technology vendors will focus on 
the importance of technology, not cultural 
transformation. Vendor interests often 
preclude reminders that KM is complex 
and cannot be accomplished through 
technology alone. 

The same is currently happening in the EP 
market. EPs are promoted as the “killer 
application” for KM and, through clever 
software vendor marketing, companies might 
be misled to think that implementing an EP is 
equal to implementing KM. 


The relationship between KM and EPs 


According to Shilakes and Tylman (1998) of 
Merrill Lynch’s Enterprise Software Team: 


Enterprise Portals are applications that enable 
companies to unlock internally and externally 

stored information, and provide users a single 

gateway to personalised information needed to 
make informed business decisions. 


It is interesting to note that some of the very 
first research published on the topic of portals 
refers to the term “knowledge portal”, for 
instance a Gartner Research Note entitled: 
“The knowledge portal: adding knowledge to 
intranets”, 30 December 1998 (Bair, 1998). 
It seems that right from the start, portals were 
connected to KM. 

In current literature, the relationship 
between KM and EPs is described primarily 
as an IT relationship. For instance, Gartner 
claims: 
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The overlap between current KM and EP 
technologies is not surprising since they share 
the same parentage — the Internet ... . 


Gartner continues to say that although KM 
has been promoted as a business management 
issue for more than 20 years, the Internet was 
the technology trigger that launched KM into 
worldwide adoption. EPs were also triggered 
by the Internet, as enterprises sought to 
replicate the Internet portal (e.g. Yahoo!) 
within the enterprise to unify information 
access and improve management of vast 
information resources (Harris et al., 1999b). 

Another aspect of the IT relationship is seen 
in the fact that EPs are the value proposition 
first used by groupware systems such as Lotus 
Notes, and more recently, corporate intranets 
— which are both recognised KM tools. For 
many, the portal approach merely seems like 
the natural evolution of intranets and 
groupware solutions into a common 
information infrastructure (Computer Finance, 
2001; Watson, 2000). 

Gartner (Harris et al., 1999a) provides a 
checklist of KM technology requirements and 
identifies EP’s overlap. In Table I the 
significant overlap between EP and KM 
technology requirements can be seen, making 
EPs recognised KM tools. 

In addition to the technical similarities, 
there are also a few authors who regard the 
relationship between KM and EPs as being on 
a more strategic level. For instance, Meta 
Group says EPs are foremost a business 
strategy, not a technology endeavour, and 
that the core value of EPs is information 
dissemination. In a KM context, the EP’s 
main role is to provide easy access to the 
shared services and knowledge resources that 
constitute a network-based corporate 
memory. For knowledge workers it provides a 
way to see and live in the corporate network 
(Cain, 1999; Computer Finance, 2001). 

Wells et al. (2000) also regard the current 
hype around portal software and applications 
as interesting since portals were originally 
conceived to address the cognitive (not 
technical) problem of providing easier access 
to information, especially on the Internet. 
The spread of EPs promises to bring a sense 
of order to corporate information repositories 
similar to that which consumer Internet 
portals have brought to the World Wide Web. 
Wells et al. (2000) therefore see the 
relationship not as technical but as strategic; 
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Table 1 KM and EP technology overlap 


Technology functionality 
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KM technology EP technology 





Capture and store 

Search and retrieve 

Send critical information to employees 
Structure and navigate 

Share and collaborate 

Synthesise 

Profile and personalise 

Solve or recommend 

Integration with existing business applications 
Maintenance 


Source: Harris et al., 1999a 


* 


* * 
* 
* * 
* 
* 
* * 
* 
* * 
* * 





KM and EPs share the same “intellectual” 
purpose, that of managing access to 
information and knowledge. 

However, although KM and EP 
applications both have rich technology 
support for information management and 
human processes and share the same strategic 
objective, implementing an EP is not 
implementing KM. Organisations cannot 
achieve the implementation of a KM 
discipline with technology alone. EPs should 
therefore not be mistaken for the essence of 
KM (Harris et al., 1999a). 


EPs are not KM 
EPs need to be implemented within a 
comprehensive KM strategy. Briggs (2000) 
says: 
A comprehensive KM strategy is the foundation 
~ and the purpose — for a successful portal. 


Companies should not confuse the tactical 
advantages offered by portals with the 
fundamental changes required for effective 
KM. The portal is the interface, the place 
where information exchange and knowledge 
transfer takes place, but it is only one 
component of successful KM (Ifrah, 2001). 
Louise Temkin (2001), Knowledge 
Operations Associate Director at Accenture, 
supports this view and illustrates the 
relationship between KM and EPs through a 
model adapted from the work of Karl-Erik 
Sveiby (1998) (see Figure 2). According to 
Temkin, the potential of an EP can be 
enhanced if it is seen to be an enabler of 
intellectual or knowledge capital management 
which is directly linked to the ultimate 
profitability or increased effectiveness of an 
organisation. The EP links the different silos 


of knowledge capital with each other as well 

as with the value proposition and 

organisational culture of the enterprise. 

As is shown in Figure 2, KM facilitates the 
creation, capturing, organisation, accessing 
and use of an enterprise’s knowledge capital, 
consisting of: 

+ Human capital — knowledge, skills, 
experience, talent, competencies, 
education, intelligence, innovation, 
creativity and problem-solving ability of 
the people that work within, or in 
partnership with, an organisation. 

* Structural capital — systems, processes, 
methods, best practices, know-how, 
agreements, copyrights, patents, trade 
secrets, brands and techniques which get 
left behind when the employees leave. 

* Relational capital — relationship with 
customers, suppliers and external 
organisations. 


Value is created by personalising, aggregating 
and integrating the human, structural and 
relational capital. The value of KM should be 


Figure 2 Model linking KM and the EP 
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unique and specific to every organisation and 
should be reflected in the bottom line. An 
organisational culture should be created that 
incorporates KM, including motivation, 
ability, performance, education, learning, 
training, trust, behaviour, values and beliefs 
(Morris, 1998; Sveiby, 1998; Temkin, 
2001). 

The EP facilitates human capital creation 
by consolidating, integrating, enhancing and 
connecting to the knowledge people have 
within an organisation. It enables structural 
capital to be shared, used, re-used, acquired, 
retained, identified, received, transmitted, 
applied and networked within an 
organisation. The EP facilitates, collaborates, 
updates, explores, interacts and promotes the 
image and the brand to customers, partners 
and interested parties (relational capital). An 
organisational culture should be created in 
which the portal is used. For this reason the 
portal strategy cannot be stand-alone, it must 
be incorporated into the organisational 
strategy for knowledge capital management 
(Temkin, 2001). 


Case study 


In order to test the theory, a case study will be 
described where an EP has been implemented 
in a global IT consultancy. Since the EP is 
seen as a KM tool, the basic components of a 
KM implementation as described in Figure 1 
should be present. 


Case study: information technology 
consultancy 

The case study organisation is a global IT 
consultancy with 78,000 employees all of 
which have intranet access. The majority of 
the employees work at client sites or remotely 
from their local offices and depend on the 
organisation’s intranet infrastructure to help 
them manage their administrative and 
knowledge related tasks. 

In 2001, as part of its IT infrastructure 
strategy, this IT consultancy designed and 
developed an EP. The EP provides a global, 
single point of entry to the organisation’s 
knowledge, information and tools, in order to 
assist its employees in their day-to-day tasks. 

The company’s KM efforts are initiated to 
leverage the knowledge capital of a large, 
global, multidisciplinary organisation. 
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Knowledge sharing is at the core of the 
programme, and the relevant KM solutions 
are provided to capture and organise its 
collective knowledge assets. All employees 
practice KM in that they use the knowledge 
environment as a touch point to collaborate 
with experts, to access relevant knowledge, to 
speed up project work and to resolve issues. 
In addition, knowledge creation is 
emphasised in the expectation that all 
employees contribute their own experience 
and expertise to the collective knowledge 
base, and that they also utilise it to stimulate 
ideas for problem solutions, new services and 
new business. 

The company’s KM solutions were initially 
built on Lotus-based KM technology 
architecture, but it is gradually moving to 
increased browser-based access. The 
Web-based EP will eventually become the 
exclusive KM tool in the company. The 
design, development and implementation of 
the EP were driven by a project called 
“Project Navigator”, which was jointly 
sponsored by the IT team and the Media & 
Entertainment industry group. The EP 
governance team included senior managers 
from all areas of the company, e.g. business 
architecture, IT, KM and the business and 
support units. The involvement of senior 
leadership provided executive sponsorship to 
reinforce the usage of the EP. The channel 
management team created “portal guiding 
principles” that were used in the design and 
development of the EP. 

The EP was launched as a pilot during May 
2001 and the official global launch to all 
company employees took place during July 
2001, over a two-week period. The target 
audience for the initial launch of the EP was 
line consultants with the intent of making 
them more effective and efficient in delivering 
value to their clients. 

The implementation of the EP was based 
on the framework depicted in Figure 3, with 
the key components being strategy, 
organisation/culture, measure, technology 
and content architecture. The heart of the 
model is helping the employee perform their 
business processes more effectively. 

Phase 1 of the EP was an enhanced 
directory of available sites that was easy to 
navigate and which provided a sophisticated 
search engine that would assist users in 
accessing and browsing specific pieces of 
information. With the implementation of 
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Web publishing and content management, as 
part of Phase 2, the EP has been able to 
integrate a number of sites into its overall 
structure and has built robust channel 
structure to access pertinent information. In 
addition, it has a home page that can be 
customised by geography to deliver local 
information of interest. 

The EP content includes: information on 
the company, knowledge capital (e.g. project 
deliverables) that is also available on Lotus 
Notes; news; learning and training available in 
the company; tools and services; travel 
information; and career and benefit 
information. Currently the EP contains the 
KM functionalities listed in Table II. 

Future enhancements to the EP include: 
personalisation, upgrade of the technical 
infrastructure to include a content 
management platform, integration of basic 
transaction processing and further integration 
with other enterprise applications. 

KM in the company is embedded within the 
human resource function. This means that, 
included in employees’ performance 
evaluation, is the measurement of their 
contribution of knowledge capital to the 
Lotus Notes databases or EP. 

Despite the strong KM culture in the firm, 
the uptake of the EP as the new KM tool is 
still relatively low. Some of the reasons 
include: 

* Most important is the fact that the Lotus 
Notes databases still exist. Employees are 
more familiar with and aware of this tool 


Figure 3 EP framework 
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and will continue using it until such time as 
Lotus Notes has been replaced by the EP. 

+ The EP is plagued by various technical 
problems including down-time and 
extremely slow connection time when 
working from client sites. Information 
contributed to the EP gets “lost” and 
there are still problems with the 
automatic updating of information from 
Lotus Notes to the EP. 

+ The ownership of the content on the EP 
is fragmented and there is a silo effect. 
This results in varying data quality; for 
instance certain databases have been 
transferred completely to the EP and 
others not. In the latter instances 
employees still need to refer to the Lotus 
Notes database to be able to access all 
available information. 

+ The silo effect is also emphasised by the 
fact that moving from one section in the 
EP to another requires a continued input 
of an employee’s enterprise password. 

¢ Contributing knowledge capital in the 
form of project deliverables, best 
practices, proposals, procedures, etc. to 
the EP needs to be done during a 
consultant’s “spare time”, since it is a 
non-billable task. These contributions 
therefore take a low priority in 
consultants’ daily work, resulting in a loss 
of valuable knowledge capital. 


The success of the EP in this company will be 
measured in terms of the impact it makes on 
the bottom line, in other words the impact on 
the return on investment. In terms of the 
future of the EP, an announcement was made 
within the company that the EP would 
completely replace Lotus Notes during 2003, 
becoming the primary KM tool. 


Case study — best practices 

Although the EP discussed is in the early 

phases of implementation, certain best 

practices can already be identified in 

implementing EPs. Some of these best 

practices include: 

* When determining scope, identify the key 
capabilities that will drive repeat usage. 

+ Build the site with the goal of creating a 
tool that is useful initially, and then add 
features and information to make it 
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Table ll Consultancy EP's KM functionalities 


KM functionality 


Description of EP 





Knowledge/information 
capturing 

Expert system 

Application integration 


Communities of practice 

Personalisation 

E-learning 

Automatic update of 
content created through 
linked applications 

Browsing and full text 
searching 


Structured and 
unstructured 
information 





Knowledge capital can be contributed cirectly to the EP as of April 2002 through standardised templates. Contribi 
can be made by non-technical users 

The EP contains an “expert finder” functionality 

The EP integrates a large number of Lotus Notes databases as well as other company sites. Applications not yet 
integrated include, for instance, the performance evaluation tools and office-specific directories 

Employees can subscribe and participate in discussion forums 

Limited personalisation, for instance the system recognises recurring users, and country preferences can be selecte 

Through the EP, employees can access the eLearning database, which contains all training available in the compa: 

Knowledge capital contributed to a database in Lotus Notes is updated automatically in the EP, and the other wa 
round 


Full-text searching provides the ability tc search dozens of company sites and databases at once. This was not 
previously possible in Lotus Notes. The EP has a standard interface and navigation scheme with logical organis: 
and placement of content 

Knowledge capital, especially in the form of human and structural capital, is captured on the EP, including best 
practices, client deliverables, processes created within projects, proposals, client presentations, etc. There exists 
logical organisation and placement of information 











essential, so that it eventually becomes 
the user’s workspace. 

* Clearly understand the needs of the 
end-users upfront by conducting focus 
groups. Continue to drive decisions based 
on user input. 

* Clearly establish the target audience 
initially; do not attempt to be all things to 
all people. l 

+ Attempt to get executive sponsorship to 
reinforce the portal as the one portal 
recognised by leadership. 

* Organisational and cultural issues to be 
addressed include data ownership and 
encouraging employees to share and 
contribute to the corporate data pool. 

* Focus on incrementally adding 
communities rather than pursuing a 
big-bang approach for the whole 
organisation. Each community will 
require attention and specialisation that 
could make an all-at-once 
implementation daunting. 


Conclusion 


From the case study it is clear that EPs are 
unique and will differ according to the specific 
objectives and characteristics of an 
organisation. The nature and value of KM are 
also unique in different organisations, 
according to the type of knowledge capital 


available in the organisation, as was suggested 
in Figure 2. 

Despite these differences, the theory 
highlighted the fact that implementing an EP 
requires a complete KM strategy, which will 
have the same basic elements regardless of the 
type of organisation. In order to determine 
the degree to which the implementation of the 
EP in the case study was done within a KM 
strategy, the EP will be evaluated against the 
basic eléments identified in the KM model as 
described in Figure 1. $ 

The first dimension of the model, consisting 
of knowledge creation and innovation 
activities, was present in the case study 
organisation with the focus on knowledge 
capture and the “codification” of knowledge. 
It appears that the more mature and 
established these basic KM activities are 
within an organisation, the more effective the 
EP will be in achieving the necessary objectives 
in terms of sharing and providing access to 
knowledge and information. It supports the 
theory that EPs are therefore not a substitute 
for poor KM. Successful KM will lead to a 
successful EP, not the other way around. 

In terms of the second dimension of the 
model, consisting of those elements that 
enable the knowledge-creation activities, 
Table III summarises to what degree these 
elements were present in the case study 
organisation. 

Two issues emerge from this evaluation. In 
the first place it is clear that there still exist 
huge differences in opinion in terms of what 
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Table III EP alignment with KM elements 








The EP was implemented as part of the organisation’s IT strategy. In the organisation there is a clear alignment of the EP 
strategies with the objectives of the organisation; the EP supports the core business activities of the organisation. The EP 


Benefits achieved through the EP, in terms of improved access to and sharing of organisational knowledge and information, will 
be determined informally in the organisation. On a more direct level the organisation will measure the use of their EP through 


No formal or written policy exists for information and knowledge management through the EP in the organisation. However, the 
organisation has adopted and communicated this information sharing policy on a high level to employees 

It seems that the majority of KM initiatives in the organisation revolve around identifying, classifying and indexing structured. 
knowledge and information on the EP. There is a strong drive to capture human and structural capital on the EP 


The EP in the organisation is in the process of creating a technology environment where processes to achieve the organisation's 
objectives will be automated and where knowledge and information collaboration will be supported. However, as was 
mentioned in the second “best practice”, the EP in the organisation is currently useful, but it is not yet at a level where it is 


KM element Case study 
+Strategy 
strategy is aligned with the organisational KM strategy 
Measurement 
employee performance evaluations 
Policy 
Content 
2 (see Figure 2) 
Process 
regarded as essential — employees do not yet “live” in the EP 
Culture 


In the organisation there is very limited time and energy spent on creating and promoting a KM culture, in order to encourege 
knowledge, information sharing and collaboration across and among employees/business units through the EP. Since the EP is 
in the early stages of implementation and rollout, the focus is currently on technology and design issues 














an EP could or should be. The strong link 
between KM and EPs as found in the 
literature is also not necessarily seen in 
practice. At these early stages it seems that 
organisations use EPs as content and 
information management tools, and only to a 
limited extent as true KM tools. 

In the second place, and following from the 
above, it is clear that the focus is on the EP 
technology and the design and development 
of the tool. There are limited concrete 
processes in place to create a culture where 
employees are encouraged and supported to 
share and re-use knowledge, in general and 
through the EP. 

In conclusion, organisations that adopt EPs 
as KM tools should take account of all 
elements of KM, including human processes 
and culture; otherwise they will eventually 
find that they are effectively “all dressed up 
with nowhere to go” — they have a 
“knowledge repository” that decays through 
disuse. 
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Introduction 


Knowledge management (KM) as an 
identifiable area of study and practice is often 
thought to have first emerged in 1995 with 
Nonaka and Takeuchi’s The Knowledge 
Creating Company, although earlier references 
to the “knowledge worker” can be traced to 
Peter Drucker (1983): 

To make knowledge work productive is the great 

management task of this century; just as to make 


manual work productive was the great 
management task of the last century. 


In some sectors, KM has been adopted 
quickly and with enthusiasm, in consultancy 
firms (applying it to themselves, then selling it 
as a service), energy (oil and gas) and 
pharmaceuticals in particular. Take up has 
been slower in the public sector, although its 
use is on the increase in government 
departments in both the UK and Australia. 
The latter two countries, independently, are 
at the forefront of seeking to create standards 
and guidelines (Kelleher and Levene, 2001). 

KM is inherently cross disciplinary and 
cross professional, with implications for line 
management but also computing, human 
resource and library professionals. The 
starting point of the knowledge library 
(KLibrary) project was the reflection that KM 
has been taken up very differently in different 
library sectors, and that this should be 
explored. 

KM authors sometimes see librarians as key 
brokers of knowledge sharing. Davenport and 
Prusak (1998), for example, recognise the 
possibility that librarians’ knowledge of who is 
researching what enables them to connect 
people in different parts of the organisation, 
often in unexpected ways (Davenport and 
Prusak, 1998, pp. 29-30). Generally, though, 
most library work would be seen by KM as 
primarily in the realm of managing explicit 
knowledge (which is assumed to be well 
understood) rather than the management of 
intangible, tacit knowledge which is often the 
focus of KM interest. 
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In turn, knowledge management has had a 
mixed reception from the information 
profession. By reputation KM has been 
quickly adopted in commercial operations 
and law firms, and in some of these cases the 
library has been heavily involved. Despite the 
strong advocacy of KM in the profession by 
professional bodies such as the Library 
Association, now CILIP (Field, 2002), in 
most library sectors there has been little take 
up of KM to date. 

Some in libraries have been critical of the 
whole idea as a mere management fad (e.g. 
Bawden, 1998). Those aspects of KM close to 
information management have been criticised 
as ill informed attempts to reinvent library 
knowledge about controlled vocabulary and 
metadata. They have also been perceived as 
no more than a return to a focus on 
organisational skills inventories and expertise 
databases, an emphasis that has never been 
lost in special libraries (Streatfield and 
Wilson, 1999). KM technologies so often 
turn out to be reapplications of indexing and 
portal technology that it is easy to dismiss KM 
as a rebranding exercise. The impact of KM is 
further obscured by KM itself being a notably 
fuzzy concept. 

Again, in some settings where KM is on the 
institutional political agenda, it seems that 
appropriating the term “knowledge” is merely 
a resource winning strategy. For example, to a 
certain extent in the health field the mantra 
“access to the knowledge base” can just mean 
access to Medline. In a context where the 
loosely defined term knowledge has become a 
government declared priority it is good in 
itself that it is being used to boost the 
perceived value of traditional information 
assets, but this is not to connect library 
practice and theory and KM. 

Again there is interest among academic 
librarians in KM concepts, but not much 
evidence of use of KM terminology in practice 
or in the literature. This may be because the 
scholarly community is already the model of a 
knowledge orientated society, but it is 
surprising that more of the terminology of 
KM has not percolated into day-to-day 
practice in university libraries. 

This may be just a matter of terminology. 
Many existing library practices can be seen as 
KM, for example people finders/yellow 
pages/expertise databases or SDI services. In 
that sense librarianship teaches some forms of 
KM already. 
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Research approach 


The Knowledge Library Project started from 
the premise that it would be useful to 
assemble a group of librarians from across the 
library sectors to share their understanding of 
the concept of KM, and to look at how some 
of the ideas from the rich literature of KM 
are, or could be, applied to library practice. 
For the project, two particular notions in the 
KM literature were identified that did not 
seem to be well understood by librarians, and 
which could be the focus for thinking about 
how to expand library theory and practice: 
community building; and storytelling. 

It was considered that the most suitable 
research approach for the project to adopt 
would be that of action research. This was 
because the project was both about 
understanding and establishing the potential 
role of KM for librarians and also 
encouraging its use. Action research involves 
collaboration between the researcher and 
practitioner, with the aim of bringing about a 
change in a particular situation, to effect some 
improvement, in this case to increase 
awareness and knowledge of KM. The project 
team determined also to use an Action 
Learning approach, as it was intended to take 
learning from the face-to-face participatory 
seminars into shaping the details of the online 
phase of the project. Action Learning is where 
“... work and learning occur simultaneously” 
(Weinstein, 1995), with activity addressing 
work-based issues, e.g. applying KM in the 
librarians’/information professionals’ working 
situation. However, with the benefit of 
hindsight it is possible that a journey-making 
(Eden and Ackerman, 2001) approach would 
have proved to be more suitable, enabling 
more active and extensive action learning. 


Community 

Though a rather inflated term for which social 
network could be substituted, community is a 
core notion in KM. Kim (2000), defines an 
online community as: 


... a group of people with a common purpose, 
interest, or activity, who get to know each other 
better over time. 


In KM communities are important as groups 
of people who share ideas and insights, help 
each other solve problems and develop a 
common practice or approach. They are not a 
new kind of organisational unit; rather, the 
stress on community represents a new 
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discovery of the value of relatively informal 
interactions. 

Communities are significant forces for the 
sharing of knowledge within organisations, 
and between individuals and groups. 

At the heart of all communities can be 
found “social capital”, which Putnam (1995) 
described as the: 


... features of social organisations such as 
networks, norms, and social trust that facilitate 
co-ordination and co-operation for mutual 
benefit. 


For Lesser (2000) it can be seen: 


... as the wealth (or benefit) that exists because 
of an individual’s social relationships ... 


and Prusak (2000) the: 


... value that accrues to an individual or 
organisation through who they know and how 
they use it (sic). 


Cohen and Prusak (2001) in reviewing the 
many definitions concluded that: 


Social capital consists of the stock of active 
connections amongst people: the trust, mutual 
understanding, and shared values and 
behaviours that bind the members of human 
networks and communities and make 
co-operative action possible. 


Types of online community 

The different types of online community can 

be broken down by the purpose, and shared 

characteristics of their members: . 

* Communities of practice (also known as 
CoPs) — where individuals share the same 
profession, such as Accounting WEB, an 
online community for accountants. 

* Communities of circumstance — where 

individuals share a personal situation 

such as Square Canary, a community for 
people who work in and around Canary 

Wharf in the City of London. 

* Communities of purpose - where 
individuals share a common objective or 
purpose, such as KnowledgeBoard, a 
community to define knowledge 
management “made in Europe”. 

* Communities of interest — where individuals 
share an interest, such as UK-Dance a 
community for people interested in dance 
music in the UK. 


Sometimes a community may fall into more 
than one definition, and over time a 
community may develop sub-communities 
formed around special interest groups 
(Baxter, 2002). There is also a new 
generation of corporate communities 
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emerging, as companies are recognising the 
power of community to strengthen internal 
and external relationships (Baxter, 2002). 

In their latest work, Wenger et al. (2002) 
see a librarian role in a community of 
practice. If such communities are the key to 
knowledge creation and transfer, the library 
profession may be ambitious to increase its 
skill set to become the natural manager of 
community. The challenges are in fact exactly 
those the profession has faced in trying to 
persuade people to use intranets. And it is 
evident from job adverts that there is an 
emergent professional group of community 
moderators or facilitators who might find a 
home within the professional society of 
librarians, CILIP. At the very least it would be 
useful to identify more clearly the role of 
librarians in educational communities of 
practice, for example. At the same time it was 
our perception from library-orientated lists 
that they were heavily orientated towards 
information exchange and that discussion is 
muted. This could reflect a rational response 
to the limitations of e-mail as a medium for 
discussion. But it could be that core 
professional values inhibit a stress on 
discussion and debate, thus weakening their 
claim to be natural leaders in: community 
building. 

Community was also thought to be a good 
subject to discuss in the project since 
participants and researchers could reflect . 
directly on their immediate experience of the 
project community itself, reinforcing the KM 
principle of active learning. 


Storytelling/narrative 


The other KM concept that was felt it would 
be useful to explore in the project was 
storytelling, also referred to as narrative. 
Cohan and Shires (1988) state that “a 
narrative recounts a story, a series of events in 
a temporal sequence” and suggest that 
“stories structure the meaning by which a 
culture lives”. It could take the form of a line 
manager (storyteller) giving advice to a 
trainee (listener) or simply of colleagues 
talking during coffee breaks (Kelleher and 
Levene, 2001). Storytelling is, primarily, a 
tool used in cultural change. 

A number of authors, including Snowden 
(2000) of IBM and the IKM, Denning 
(2000), World Bank, and Gabriel (2000), 
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have, based on their consulting practice, 
suggested that storytelling is an effective 
mechanism for transferring tacit knowledge 
and introducing innovation. They claim that 
storytelling can be used for organisational 
culture analysis, introduction of change/ 
innovation, behaviour reengineering, etc. — in 
essence storytelling as a knowledge disclosure 
mechanism. Stories are a powerful 
communication tool that can evoke visual 
image and emotion, whilst at the same time 
allowing information and knowledge shared 
to become more realistic and easy to accept 
(Morgan and Dennehy, 1997). 


Nothing serves a leader better than a knack for 
narrative. Stories anoint role models, impart 
values, and show how to execute indescribably 
complex tasks (Stewart, 1998). 


Storytelling represents an “old skill in a new 
context” (Snowden, 2000). Narrative and 
story act as a form of knowledge holder in its 
representation of events, this can be in varied 
forms: oral; written; film/video; or even 
graphic (pictures and maps). Narratives and 
stories can be seen to provide a bridge 
between the notions of tacit and the explicit 
knowledge, allowing the demonstration and 
sharing. 

Story provides a simple agent through which we 


can communicate complex meaning (Snowden, 
1999). 


In fact, storytelling complements community 
building (Wenger, 2001) in promoting 
organisational sharing, continuous learning 
(Morgan and Dennehy, 1997), cultural 
changes (Jordan, 1996) and above all the 
recognition of people within the organisation. 
Denning (2000) suggests that storytelling is a 
collective process. Some of the first writing 
about communities of practice saw 
storytelling as one of its key defining 
attributes (Brown and Diguid, 1991). 


Knowledge Library Project 


The object of the project was to begin to do 
the early work in forming a network of 
librarians interested in KM. The network 
would be a place to learn more about KM 
from South Bank University and guest experts 
but increasingly from other group members. 
The creation of a small community in the 
project itself would offer immediate practical 
experience of being involved in such a 
network, and planned joint storytelling 
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activities were organized. The researchers 
wanted members to reflect on their practice to 
see how far KM concepts could clarify things 
they already did, or contribute to the 
widening of their skills. 

Within the brief time scale of the funded 
project (eight months) it was only likely to be 
possible to dig the ground and sow the seed. 
Community building is a long-term process 
and it was expected to take the group only to 
the point of “coalescing” as a group (Wenger 
et al., 2002, p. 69). The purpose of the project 
was to make such a beginning, with further 
work linked to other KM initiatives by the 
researchers. 


Participants 


Around 20 participants from special, law, 
academic and health libraries volunteered to 
join the group in response to direct invitation 
and invitations on a small number of 
discussion lists. 

An initial survey established that the 
participants had a very wide variety of views of 
knowledge, but most (15/16) saw knowledge 
as very important to their organisation and a 
key responsibility for them as individuals 
(16/16). They were much more divided about 
whether their organisation was good at 
sharing knowledge (six said yes). 


Seminar phase 

Over the next three months the project team 
organised three workshops to introduce the 
members to the team and each other, and to 
reinforce briefing papers published on the 
project Web site. 

The first meeting was centred around a 
series of talks from South Bank University’s 
School of Computing, Information Systems, 
and Mathematics (SCISM) on aspects of 
KM, with associated exercises and 
discussion[1]. It was difficult to gauge the 
level of these talks, given participants’ 
different backgrounds and levels of prior 
knowledge. The project team’s immediate 
feeling after the session was that they had not 
done enough to create a sense of commonalty 
between the members, and that they had 
reinforced a sense that it was a course on KM, 
whereas they wanted to stress learning from 
each other and reflection on existing practice. 
In partial response to this a Jiscmail 
discussion list was created, as an outlet for 
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participants to continue to talk to each other 

between the workshops. 

The second seminar was much more 
discussion based, and organised around a 
series of reflective exercises. The first asked 
participants to reflect on: 

+ What were they actually sharing and 
whether it represents information or 
knowledge? 

+ What is the form of this sharing, i.e. 
document, conversation, or 
recommendation? 

* How is it facilitated, i.e. person to person, 
or via technology? 


The second exercise was to look at what 
barriers they have encountered, and how they 
have or could be overcome. The third focused 
down on thinking about their own experience 
of online communities, and how useful it had 
been for them. 

Reviewing these discussions, the project 
team felt that there was a strong sense that 
knowledge practice is in many ways closely 
linked to library practice. But there was a 
strong anti-technology feeling in the group, 
slightly surprising given the success of the 
library profession at adapting to the 
opportunities created by the Internet, for 
example. 

Between the second and third workshops 
the research team evaluated a number of 
software tools for community building. As 
part of the process the participants in the 
project were asked themselves to rank 
different software functions for potential 
value. Real time chat was rated low. The 
discussion area was seen as the most valuable, 
along with the directory of members. The tool 
devised to evaluate available software systems 
is to be published separately. 

The third seminar started with a 
demonstration of the chosen system, 
intranets.com, supported by a manual written 
by the project research associate (RA) 
followed by an opportunity to try the system, 
by navigating through sections and posting 
messages. The second part of the session was 
a storytelling workshop led by Selina Shah, 
Senior Knowledge Management Officer, The 
Arts Council. Participants, who were initially 
dubious, seemed captivated by the idea and 
joined in the storytelling exercises with 
enthusiasm. Props were provided (such as 
coloured pens, coloured dough, scissors, 
hats) and books were utilised, and 
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participants were getting to grips with 
presenting their individual stories. However, 
it was apparent that for this technique to 
work it would require a lot of skills and 
confidence on the part of the storyteller. 
Language, communication and listening were 
some of the skills identified. This was seen as 
one of the initial barriers for participants to 
apply this technique to their own settings; 
however, people were keen to try it out. The 
project team was also keen to transfer what 
was learnt from the workshop to the online 
community. 

Materials from the workshops can be found 
on the project Web site at (see Figure 1) 
http://www.litc.sbu.ac.uk/klibrary 


Online phase 


After the third workshop communication was 
to be through an online system. The software 
chosen had the following functions: ability of 
administrators to make announcements, all 
members to start discussions and post to 
discussions, ability for all members to start 
and vote in a poll, ability of all to add links or 
documents, ability of all members to add 
events and tasks with reminders to a calendar, 
ability of individuals to create a membership 
directory entry for themselves from a 
structured template. 

The online community system was run for 
three months. In retrospect, activity fell into 
two periods. In the first period, the link and 
document stores on the site were primed with 
content. A number of discussions were 
initiated, and the team tried out a range of 
type of postings, from questions to posting 
pieces of text, to see which would be most 
successful in eliciting a response. The alert 
capability was used to send mails to announce 
and encourage participation. A series of polls 
were conducted on attitudes such as how to 
define membership of a community and on 
issues such as whether guest members should 
be allowed to join. The community as a whole 
voted not to open the site up to 
non-members. 

By the end of the period most of the 
members had registered on the system. But 
only one thread of discussion had been started 
by members other than the project team. Only 
a handful of members ever posted a question 
or answer. Most log ons were stimulated by 
an alert sent out by the team. 
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Figure 1 Knowledge library Web site 
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The project team left the site alone for a quiet 

period to see if inactivity would stimulate a 

more active response from the participants. 

This did not happen. 

In the second period, after about six weeks, 
the researchers tried to focus activity by 
archiving past discussions and creating a 
limited number of new discussion areas (even 
though this risked reinforcing the idea that the 
researchers were “in control”). 

The four areas were: 

(1) Own issues area — where people were 
invited to send specific issues for 
discussion. 

(2) The klibrary story — this was a chain 
storytelling exercise. The team first 
composed and then posted the end of the 
story to the site. Members were asked to 
take turns to write episodes, each 
contributing 100 words, nominating the 
next person in the chain. The purpose of 
the exercise was to exploit evident interest 
in storytelling to enhance participation, 
and it was thought that a story would give 
a structure to contributions. The element 
of fiction would allow a little latitude to 
express opinion without being seen to 
comment in a public way about their own 
situation. The story would also help to 





create a natural rhythm to the community 
(Wenger, 2001, p. 48). 
(3) Evaluation of software — an area to 
discuss how well the technology worked. 
(4) A “tea room” for other discussions. 


Although some personal issues were 
presented, and six episodes of the story 
written, activity from the members continued 
to be disappointing. Though they continued 
to log on, they did not post messages or 
resources to the site. Having reached the end 
of the funded period of the project, the 
researchers assessed progress to date and set 
out the strategy for continuing the initiative. 


Reflections on the knowledge library 
experience 


In Lave and Wenger (1991) a community of 
practice is an intangible abstraction from 
social relations, from how people naturally 
behave. But increasingly, among later writers, 
it becomes a thing which can be created and 
managed, a CoP. This divergence reflects a 
tension between sensitive explorations of how 
organisations really work, and a desire to 
derive a management formula (perhaps to sell 
consultancy and books). In fact the idea of 
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managing knowledge is itself increasingly a 
deprecated notion (e.g. Streatfield and 
Wilson, 1999, p. 70). Knowledge is too 
intangible and subtle to be managed like cans 
in a production line or books on a shelf. It is 
evident then that an attempt to create a 
community, especially in a short time scale, 
was always likely to be difficult. 

It would be fair to acknowledge that gaining 
active participation from the members of the 
knowledge library project was even more 
difficult than had, been expected. Early 
activity within the online environment was 
slow and grudging, and showed little sign of 
having a life of its own. Having identified 
personal space in the membership directory as 
a key function, few put in anything more than 
minimal contact details — even though this 
had been discussed in one of the seminars. 
Only the project team uploaded photographs 
of themselves. No one uploaded links or 
documents apart from the project team. 
Communication on the discussion list was 
dominated by the researchers. This is a 
common experience in online communities 
judging from most archives. The literature 
had already indicated to us that it would be a 
slow process. 

Reflecting on three months of experience, 
the project After Action Review identified a 
number of key issues: 

* Time. Any sort of community needs time 
to build up its own life and momentum. 

* Critical mass. The membership was quite 
smail at only around 16, and given that 
participation rates in online discussions 
always tend to be low, the project did not 
give a critical mass of people to generate 
continuous activity and interest. A small 
group had been chosen because it was felt 
it would work best in the early 
face-to-face seminars. Once the online 
community was launched the preference 
seemed to be for larger numbers. There 
was a discontinuity here. 

+  Dhversity of membership. The members 
chosen for the group did not know each 
other before the first meeting, and 
worked for different organisations in 
different library sectors. So it was a very 
diverse group and it was always likely to 
be difficult to overcome the feeling that 
they had little in common. Given that in 
the literature communities are normally 
created from people within one 
organisation, the project had added 
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another level of difficulty to the 
community building process. There was 
also a considerable variation in 
participants’ baseline knowledge of KM 
concepts. The project team felt that this 
was acceptable because it was expected 
that those with more experience would 
provide some leadership, and because, in 
fact, much of common library practice 
could be read or discussed as KM once 
the terminology was understood. 

Focus. It is possible that the project team 
had not given the community a narrow 
enough or clear enough focus. In the early 
seminar it was felt necessary to give 
everyone a basic grounding in KM 
concepts, in quite a wide-ranging way. 
This may have dissatisfied some who 
already knew much about KM, but it also 
unfocused the purpose of the community. 
It reinforced any misconception that it 
was a short course on KM, rather than an 
attempt to get librarians to reflect on their 
own practice. 

ICT. The use of ICT in the project was 
appealing, because it made the 
community building process potentially 
reusable and inherently scaleable. In fact 
many of the members expressed quite 
deep resistance to the use of ICT during 
seminar two. This is surprising among 
members of a profession that has. been 
relatively quick in adopting ICT. It may 
partly reflect the idea that posting 
something on the Web is a form of 
publication and participants were 
concerned about their expertise to do so. 
For the law librarians there were issues of 
confidentiality. It is obvious that 
face-to-face contact is much more 
successful in creating a deeper sense of 
trust; there needed to be more 
face-to-face contact. 

Incentives. Although the researchers did 
ask respondents to agree informally to 
offer a total of 24 hours of their time to 
the project during its course, there was no 
way to enforce this. It is possible that the 
participation would have been much 
greater if the participants’ employing 
organisations had been asked to make a 
formal commitment. There was little the 
team could do to incentivise 
participation, except by making it an 
inherently rewarding experience. 
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Services. The literature shows that it is 
most effective to build community by 
attracting people to use a set of subject 
based services then to build up discussion 
among the people who use the service, 
rather than start with discussion. 
Members of the Knowledge Library 
Project said that the documents put on 
the Web site were the second most 
valuable part of the site. The research 
team did not at first see this as a priority, 
and focussed rather heavily on discussion. 
Concepts of membership. One of the polls 
conducted during the research producec 
the surprising conclusion that 
participants conceived of membership as 
merely signing up to the software system, 
not logging on, let alone posting to 
discussions. This, very passive, view of 
membership could possibly reflect typical 
attitudes and values among librarians. 
Many librarian discussion lists are busy 
but primarily informational, with 
relatively little discussion. The lists are 
essentially “information 
neighbourhoods” (Burnett, 2000) that 
are used to browse for information over 
time. A core perspective in KM is that 
investing time in informal relationships 
and discussions has long term value to the 
organisation. Some organisations, as part 
of their KM commitment, have offered 
staff a percentage of time to work on their 
own projects, in the belief that this time 
will bear fruit in improved work 
performance in the long run. Many 
organisations continue to impose close 
control on how time is used. 

Technology blocks. The technology relied 
on users logging on to a Web site, whereas 
the simpler model of pushed e-mails 
requires less effort to gain participation. In 
a sense, this was the major weakness of the 
technology chosen. The best response 
received was to alerts from the system, 
which went out as e-mails reminding 
people to log on because something had 
occurred. At the same time, although this 
elicited a response, members seem to have 
found the alerts irritating. So, if everything 
in the community had been conducted 
through pushed e-mails, participants 
might have taken more interest, but 
equally they might have been irritated by 
the interruptions frequent e-mails 
represent. 
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There were other small weaknesses in the 
software such as that the alerts of new 
material were expired too quickly. Logging of 
activity was inadequate to show how the site 
was being used. It was not possible to link 
documents or links and discussion areas. The 
supplier withdrew the instant chat facility 
between the time of the project evaluating the 
software and the subscription going live. 
Overall, though, the weaknesses in the 
software were not a factor in the low activity 
in the community. 


Leadership. In a sense the group had been 
given too much leadership by the project 
team. There had not been the space or 
time for shared interests to emerge 
naturally. On the other hand, it could be 
equally argued that the team should have 
done a lot more behind the scenes to coax 
members to post to the list. There is a 
sub-conscious assumption that this is 
somehow “cheating”, but directly 
requesting input from individuals is 
arguably more than legitimate, it is 
essential (McDermott, 2000). 
Differentiation of roles of project team. It 
would have been useful to differentiate 
the three project researchers much more 
clearly so that the researchers themselves 
could continue to play out different roles 
within the discussions. Anonymous or 
pseudonymous accounts might have been 
used to increase the sense of diversity and 
activity in the discussion areas. 
Surveillance. Because it was a project, it is 
possible that participants felt under 
surveillance, which inhibited natural 
participation. One way around this might 
have been to offer the opportunity to 
make anonymous contributions. 

The results of the exit survey of participants 
were far from negative. The Web site had 
been quite heavily used. Everyone had at 
least registered for the site and most had 
logged on. One had logged on as many as 
30 times. Opinion polls had received a 
good response. 


In conclusion, the researchers decided to 
continue in the next part of the project to: 
(1) Expand membership of the group, in the 
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first instance allowing anyone from a 
participating institution to join. Then 
anyone from an open meeting on 
communities jointly organised with 
CILIP at KMEurope. 
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(2) To use the simpler push jiscmail 
technology, supplemented with material 
on the Web site, polls (which have proved 
to be a successful feature of the current 
system) from a third party service and 
with an anonymous log in. 

Plan some simple services (a monthly 
e-mail newsletter with annotated links) to 
attract more members and focus 


(3) 


discussions. 

Tighten the subject focus of the 
community, to simply community 
building. “KLibrary will be a discussion 
about how librarians can help to build 
workplace communities”. 

Offer continuing face-to-face sessions, 
where possible in the context of joint 
events. 

Reform the “constitution” of the group 
by explicitly passing joint leadership to 
other members. 


(4) 


(5) 


(6) 


Conclusion 


Parts of the library community have been 
rather slaw to take up KM ideas, yet many of 
the existing activities of librarians could be 
understood in KM terms or be reapplied from 
information management to manage 
knowledge. In government and the health 
sector the agenda is shifting to stress the term 
knowledge. TFPL research has already shown 
how IM skills fit with KM and where the 
librarian’s skill sets have to be expanded to 
capture this opportunity for the profession 
(TFPL, 2000). Morris has found that 
employers are seeking LIS professionals for 
KM posts (cited in Mendelsohn, 2001). 
Community building is a particular area of 
growth. Storytelling is an important 

concept that could enrich the practice of 
librarianship. 


Note 


1 SCISM have been running a practitioner MSc in 
Knowledge Management Systems since 2000. 
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Internet Project; 81 per cent or 997 of the 
respondents were British. The prime purpose of 
the questionnaire was to obtain information on the 
characteristics of the users of health information 
Web sites, to obtain feedback regarding for what 
they used online health sites and what were the 
perceived outcomes associated with using online 
health information. 


Characterising and profiling health Web user and 
site types: going beyond “hits” 


Paul Huntington, David Nicholas and Peter Williams 


Keywords Health services, User studies, 
Worldwide web, Customer profiling 


Identifies Internet health user types according to 
three factors: site attributes most favoured, health 
topics most sought, and the health sites they 
visited. Knowing what type of consumer uses a site 
is important, as this should inform menu structure 
and provide an understanding of why certain kinds 
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of people visit certain sites. Web sites even in the 
same field will not appeal to all users. Much of this 
differentiation will come down to design and feel of 
the site, although site attributes and information 
needs also impact here. Data were obtained from 
an online questionnaire placed on the SurgeryDoor 
Web site, a site which provides consumer health 
information. In all a total of 1,068 responses were 
received. Shows that useful groupings can indeed 
be constructed. Users were also classified 
according to additional health sites visited, also a 
preference metric, and this additionally is related to 
site attribute preferences, information needs and 
user characteristics. 


Information needs of people with multiple 
sclerosis and the implications for information 
provision based on a national UK survey 


Mark Hepworth, Janet Harrison and Nicole James 


Keywords Health education, Information, 
Personal needs, United Kingdom 


Qualitative and quantitative methods were used to 
study the information needs of people with 
multiple sclerosis (MS) in the UK. Categories of 
information need were identified; their importance 
and difficulty in obtaining them quantified. The 
research highlighted how there were significant 
differences in terms of people’s desire for the 
information, the topics in which they were 
interested and how they wanted that information. 
Information provision to people with MS was 
found to have improved dramatically over the last 
seven years, but was found to be inconsistent in 
terms of subjects covered. However, it should be 
borne in mind that at the time of diagnosis, and 
also afterwards, it may be difficult for people to 
take on board information for a variety of reasons. 
Furthermore, different individual needs mean that 
information provision is a complex task.. Many 
intervening factors can make it fail. The research 
made it clear how access to relevant information, 
provided in an appropriate way, could significantly 
improve the quality of life of the person with MS 
and that there is considerable scope for improving 
provision of information to people with MS. 


Health information on the Internet: a qualitative 
study of NHS Direct Online users 


Peter Williams, Paul Huntington and David Nicholas 


Keywords Internet, Health education, 
National Health Service, Quality, Surveys 


Examines the way the general public exploits the 
Internet for health information, the motives behind 
usage, attitudes towards issues such as quality 
concerns, and the extent to which Internet 
interventions affect the doctor-patient relationship. 
Although a questionnaire survey was used (posted 
on the NHS Direct Web site), the questions asked 
were open, and invited free-text “‘qualitative” 
answers. This method was a success in terms of the 
amount and richness of the data accrued. Results 
suggested that the Internet is exploited in a wide - 
variety of ways, by users acting in a number of roles 
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— patient, intermediary or professional. Some 
health professionals are now so comfortable with 
the Internet that they encourage Internet usage by 
their patients. Lay users demonstrated a high level 
of understanding of issues raised, and showed a 
(healthy?) scepticism regarding the information 
provided. Many users felt that their consultations 
with doctors had been enriched by Internet- 
acquired information. 


Seeking alcohol information on the Internet 
Adam N. Joinson and Phil Banyard 


Keywords Internet, Worldwide web, 
Information gathering, Health education, 
Alcoholic drinks 


It has been argued that people may be more 
willing to seek potentially threatening 
information on the Internet than they would in 
“real life”. For alcohol drinkers, potentially 
threatening information includes diagnostic 
information on the amount they drink, and 
information about the negative effects of alcohol 
consumption. In the present study, participants 
with varying levels of alcohol intake and plans for 
change chose four alcohol-related articles to read 
(from 12), using either the World Wide Web 
(WWW) or pen and paper. Results showed that 
drinkers not currently reducing their drinking 
were more likely to seek diagnostic, potentially 
threatening anti-drinking information via the 
WWW compared with when seeking paper-based 
information. Drinkers either contemplating or 
engaging in efforts to reduce their drinking 
sought pro-drinking information on the WWW 
and anti-drinking information when using pen 
and paper. The potential role of the Internet, and 
perceived anonymity, in health promotion are 
discussed. 


Availability and use of information touch-screen 
kiosks (to facilitate social inclusion) 


Markella Boudtoni 


Keywords Social alienation, Employment, 
Health education, Information, Community care, 
User studies 


The availability and the use of touch-screen 
systems for accessing a range of information in 
different settings in the UK, in particular from 
socially excluded groups, are reviewed. A 
literature review and consultations with 
stakeholders are conducted. Touch-screens are 
used widely by the Department of Works and 
Pensions; they are used for community services 
on a much smaller scale. NHS Direct provides 
the largest number of kiosks for health 
information and promotion; InTouch with 
Health, Wellpoint Health Centres, Patient 
Information for Consent Systems are some other 
examples. Some projects that target socially 
excluded groups are identified. Factors that 
determine usage and contribute to successful 
usage are also identified. The majority of 
stakeholders express the view that kiosks should 
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be available in a variety of locations, irrespective 
of their specific content and information. 


Making health information accessible to patients 
Ray Jones 


Keywords Internet, Information, 
Health education, National Health Service, 
Patients, United Kingdom 


Many NHS policy documents state the need to 
make health information accessible to all. 
Accessibility comprises physical availability and 
personal retrievability. Touch-screen public access 
kiosks have been available since the early 1990s 
and there is some evidence from a study of NHS 
Direct kiosks that they could improve physical 
access to health information. Most health centres 
have little space for longer use of computers by 
patients. Eventually home access to the Internet 
will be available to most people. In the meantime, 
computers based in public libraries offer potential 
to provide health information for patients referred 
by their GP. However, more thought is needed as 
to the format and content of the information. 
Information tailored using the patient’s medical 
record is preferred by patients and may enable 
patients to engage more social support. More 
evidence is needed from well-designed studies to 
inform the debate. 


The potential for using hospital bedside TV 
services in England to provide patients with 
access to health information 


Gavin McWhirter 


Keywords Hospitals, Television, Information, 
Patients, Health education, United Kingdom 


This article is a summary of a report which looks 
at the potential of using television sets at hospital 
bedsides to provide patients with access to health 
information. The report was jointly 
commissioned by The Department of Health 
(DoH) and National Health Service (NHS) 
Direct Online, and focused on what could be 
done using existing (or planned) Patient Power 
infrastructure. The latest generation of systems 
from all three of the main suppliers of bedside 
television service could provide access to health 
information in a variety of ways. A group meeting 
was held with key internal DoH and NHS 
stakeholders. Interviews were held with the 
main suppliers of bedside television services. 
Opinions were also taken from several 
academics specialising in Health Informatics, 
from a range of commercial health information 
providers and a wide range of NHS Trust 
representatives. Two areas where further 
research would be of use are looking at the 
opportunities around digital radio as a way of 
providing patients with health information, and 
looking at any particular issues that exist around 
communication with patients while they are 

in hospital. 
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Digital interactive television: health information 
platform of the future? 


Barrie Gunter, David Nicholas, Paul Huntington and 


- Peter Williams 


Keywords Interactive devices, Television, 
Health education, Information, Forecasting, 
User studies 


Presents research that was designed to explore the 
early take-up of a pilot digital interactive television 
(DiTV) health information service (Living Health) 
by cable television subscribers in Birmingham, 
UK. Log data indicated real-time viewing 
behaviour of subscribers to the cable network on 
which the digital health service was carried. A 
survey of users delivered a return sample of 723 
subscribers. Log data indicated an initial surge in 
use that subsided and stabilised at a lower level. 
Over one third of cable network subscribers visited 
the digital health TV site at least once during a 20- 
week pilot period. The most popularly visited 
pages concerned sex-related information. In the 
user survey, just under one in four respondents had 
reportedly used the DiTV service. The user profile 
was oriented towards older men and young adult 
women. Over eight in ten users found the service 
easy to use and understand. Over half of users 
consulted the service in connection with a GP visit. 
Some evidence emerged that use of DiTV might 
substitute for consulting a doctor among some 
users. 


Public awareness of patient rights, consent forms 
and advance statements 


Julia Dandy 
Keywords Patients, Human rights, Information 


The subject of this study is the availability of, and 
access to, information concerning individuals’ end- 
of-life health situations. The sensitive issues 
surrounding this subject are considered in detail, 
and the restraints, primarily legal, placed upon 
information availability highlighted. 
Questionnaires and interviews were used to 
determine public and professional awareness of, 
and attitudes towards, the issues and were 
compared with a similar, albeit larger, survey 
conducted by the Department of Health. 
Complementary to this fieldwork, a review of 
current and proposed clinical policies, guidelines 
and legislation and of media commentary was 
undertaken. The study revealed much public 
confusion and ignorance about how people can 
influence the medical treatment they receive when 
seriously or terminally ill. The study revealed an 
urgent need for Government initiatives to provide 
more information on patient rights in relation to 
consent forms and advance statements. 
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diabCAL: evaluating computer-aided learning for 
diabetes patient education 


Emma-fane Berridge and A.V. Roudsari 


Keywords Computer based learning, 
Diabetes, Patients, Health education, 
Function evaluation 


The design and development of a computer- 
aided learning tool for the education of patients 
with diabétes and for the training and support of 
practice nurses providing diabetes care have been 
previously reported. diabCAL is a broad and 
comprehensive learning tool, which can be 
accessed as a modular course or a desktop quick- 
reference. Evaluation is central to the design 
process and can be formative, informing design 
and development throughout the lifecycle, or 
summative, assessing the value of the finished 
product. diabCAL was subject to rigorous 
evaluation at various stages during development, 
including: verification of curriculum content and 
structure, evaluation of navigation and 
evaluation of system content. A final hybrid 
formative-summative evaluation assessed 
usability and acceptability of the system. 
Evaluations were conducted by technical 
experts and users, and findings of the final 
evaluation study, conducted by patients, 
confirmed that diabCAL was acceptable to most 
users — that is, interesting and perceived to be 
useful — even to those with little previous 
computer experience. 


The publication of cancer research papers in high 
impact journals 


Grant Lewison 


Keywords Literature, Research, Cancer, 
Journals 


Examines a set of over 27,000 UK papers in cancer 
research in order to identify the individual factors 
that influence the impact category of the journals in 
which they are published, using multiple regression 
analysis. The most important independent 
variables that have a positive effect are the numbers 
of authors and funding bodies, the research level 
(from clinical to basic), and the presence of certain 
universities, or of the USA, in the address field. 
Inter-lab co-operation was shown to have a 
negative effect on journal impact category, as was 
international co-authorship. It is because such 
partnerships usually involve more authors and 
funding for the research that they are perceived to 
lead to higher impact work. There is also a 
tendency for papers to be published in higher 
impact journals in later years, probably because of 
market forces, which means that such journals will 
tend to expand. 
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We are witnessing a digital information 
revolution of enormous scale. For many 
people the Internet, digital interactive 
television, the touch-screen kiosk and mobile 
phone have resulted in the overnight 
transformation of an information-poor world 
into an information-rich one. We have moved 
from a world in which information needs were 
rarely ever effectively met — certainly not 
without a tremendous expenditure of effort 
and cost ~ to one in which they are 
“potentially” only too easily met. This has 
created something in the way of a culture 
shock — to both information intermediary and 
end-user. 

All this is perhaps truer with regard to the 
provision of health information and advice 
than anything else. Indeed, it would be no 
exaggeration to say that health informatics, 
especially consumer health informatics, has 
pioneered the digital way. This special double 
issue of Aslib Proceedings provides proof of 
this. We publish here a dozen articles, which 
together show the leadership qualities of the 
field, its inter-disciplinarity and multi- 
disciplinarity and its methodological riches 
and robustness. 

The articles selected here discuss a wide 
range of issues, from a number of different 
perspectives, utilising several radically 
different methodologies. Needless to say, 
given this spectrum, the authors came from a 
mix of subject backgrounds. These include 
information practitioners (Dandy), academic 
researchers (Williams, Berridge and Roudsari, 
Boudioni, Huntington), University lecturers 
(Nicholas, Hepworth, Jones, Joinson), and IT 
consultants (McWhirter). Many subject 
disciplines are represented too — perhaps 
highlighting the ubiquity and the complexity 
of health information. Naturally, there is a fair 
smattering of information science specialists, 
including those whose specialisms lie in 
information needs (Nicholas, Hepworth), 
education (Williams), bibliometrics 
(Lewison) and librarianship (Dandy). 
However, also represented are health 
specialists (Jones, Berridge and Roudsari, 
Boudioni), a computer log analyst 
(Huntington), a mass communication and 
journalism expert (Gunter), and two 
psychologists Joinson and Banyard). 

Before reviewing individual papers it is 
worth highlighting the range of methodologies 
used. They cover the full spectrum, from 
in-depth interviews with key health 
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professionals (McWhirter, Dandy) and 
information users (Boudioni), to formal 
usability studies (Berridge) and statistical 
analysis (Huntington et al., Lewison). In 
between are survey findings from large-scale 
predominantly closed questionnaires 
(Nicholas et al.), free text open-ended online 
surveys (Williams et al.), psychological tests 
(Joinson and Banyard) and usability studies 
(Berridge and Roudsari). Multi-methods are 
featured by Gunter et al. (log analysis and 
questionnaires) and Dandy (library research 
and surveys). Jones, by contrast, provides an 
overview of research into the efficacy of 
providing patient information via touch- 
screen information systems. 

Nicholas and colleagues lead with a broad 
examination of health on the Net via an 
online survey of Internet users’ reported 
utilisation of the Web. The paper examines 
the characteristics of users of health 
information Web sites, for what they used 
online health sites and what were the 
perceived outcomes associated with using 
online health information. 

Huntington et al, continue the theme of 
health Web site user types. The paper 
classifies user types according to three factors: 
site attributes most favoured, health topics 
most sought, and the health sites they visited. 
Clearly knowing what type of consumer uses a 
site is important, as this should inform menu 
structure and provide an understanding of 
why certain kinds of people visit certain sites. 
Web sites even in the same field will not 
appeal to all users. Much of this 
differentiation will come down to design and 
feel of the site, although site attributes and 
information needs also impact. 

A predominant theme throughout many 
articles published here is that of information 
needs and how current technological systems 
can help meet these. Hepworth et al., for 
example, discuss the information needs of 
people with MS and how these might be met 
using particular Web authoring tools. They 
found that needs are multifarious, and that 
some individuals do not see information as 
the panacea to their problems. This is a useful 
wake-up call for the information specialists. 
In their study of the use of the Internet for 
health information, Williams et al. found a 
huge variation in reasons for going to the Net 
and in the roles of people who do so (i.e. as 
patient, intermediary professional or for 
general interest). Often users would be 
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seeking information from a number of . 
perspectives during the same search session. 
Joinson and Banyard look at information 

seeking and use from the discipline of 
psychology. The authors argue that people 
may be more willing to seek potentially 
threatening information on the Internet than 
they would in “real life”. In the study 
reported, participants with varying levels of 
alcohol intake and plans for change chose four 
alcohol-related articles to read, using either 
the World Wide Web (WWW) or pen and 
paper. Results showed that drinkers not 
currently reducing their drinking were more 
likely to seek diagnostic, potentially 
threatening anti-drinking information via the 
WWW compared with when seeking paper- 
based information. Results suggested that 
there seems to be an effect of medium on 
information seeking — with modes of 
information seeking having a level of 
identifiability attached, people seem to 
behave in a socially desirable manner (i.e. 
seeking anti-drinking information). Second, 
the current health behaviour and plans for 
change lead to different information-seeking 
motivations, which may well also interact with 
medium. 

Many of the contributors look at the other 
side of the “needs” coin — how to provide 
most effectively the information that is 
required. Boudioni reviews the availability 
and the use of touch-screen systems for 
accessing a range of information in different 
settings, in particular from socially excluded 
groups. She also identifies factors, which 
determine usage and contribute to successful 
usage. These include a simple interface that 
takes into account the computer skills and the 
educational level of possible users, and a 
screen design to allow for easy navigation 
within and between menu hierarchies. Jones’s 
research indicates that more thought is 
needed with regard to the format and content 
of the information and to providing 
customised information based on patient 
records. 

McWhirter looks at the potential of using 
television sets at hospital bedsides to provide 
patients with access to health information. 
The author admits that it is not clear what 
sort of appetite patients have for consuming 
and absorbing health information while in a 
hospital bed, and what difference it will make 
if the information is provided in text, audio or 
video form. The interesting conclusion 
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reached is that radio might be a more effective 
and efficient way of providing people with 
access to health information than as text or by 
using videos. 

Gunter and colleagues present research 
exploring the early take-up of a pilot digital 
interactive television (DiTV) health 
information service (Living Health). Over one 
third of cable network subscribers visited the 
digital health TV site at least once during a 
20-week pilot period. The user profile was 
oriented towards older men and young adult 
women. Use of the service was predicted by 
users having a young child, being in a low- 
income neighbourhood, and having a special 
interest in a health condition. Over half of 
users consulted the service in connection with 
a GP visit. The Department of Health in the 
UK has made a major investment in testing 
the potential of DiTV as a health information 
platform. The early indications are that this is 
a platform with promise. Longer-term 
commitment and observation will indicate 
how effective this technology can be in 
relation to a range of different health 
information and advisory functions and how 
best it can be utilised to benefit public health 
care and the health profession. 

Dandy examines the availability of, and 
access to, information concerning individuals’ 
end-of-life health situations. The study 
revealed much public confusion and 
ignorance about how people can influence the 
medical treatment they receive when seriously 
or terminally ill. She suggests that there was 
an urgent need for Government initiatives to 
provide more information on patient rights in 
relation to consent forms and advance 
statements. 

Berridge and Roudsari take yet another 
angle on the theme of health information 
seeking .and provision. Their paper evaluates 
the usability of a computer-aided learning tool 
for the education of patients with diabetes — 
diabCAL. This was subject to rigorous 
evaluation at various stages during 
development, including an evaluation of 
navigation and evaluation of system content. 
Tests were conducted by technical experts 
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and users, and findings of the final evaluation 
study, conducted by patients, confirmed that 
diabCAL was acceptable to most users — that 
is, interesting and perceived to be useful — 
even to these with little previous computer 
experience. 

Lewison shows the unique contribution of 
bibliometrics to the field in a paper that 
examines UK papers in cancer research in 
order to identify the individual factors that 
influence the impact category of the journals 
in which they are published, using multiple 
regression analysis. 

Finally, drawing all these studies, reports 
and commentaries together, there are a 
number of important issues. First, 
information need is complex, with people 
seeking information for a variety of purposes 
and in a number of different roles (Williams et 
al.). A salutary lesson is that, for many people, 
this activity is not the No. 1 priority with 
regard to their health (Hepworth et al.). 
Seeking information depends upon a number 
of issues, such as perceived privacy (Joinson 
and Banyard), availability and clarity of 
information (Dandy), navigating systems 
(Boudioni). Some optimism was expressed, 
however. Digital TV seems to be reaching 
groups of health consumers that other 
electronic systems miss (Gunter et al.) and 
other initiatives, such as hospital bedside 
services and more widely deployed touch- 
screen systems, may extend the reach yet 
further (McWhirter, Jones). Berridge and 
Roudsari show that careful formative testing 
and involvement of (potential) users lead to 
excellent systems, easily accessible, which 
meet real needs. In sum, the papers offer 
much food for thought (and action), some 
optimism, and many questions. We hope they 
prove useful. 

All papers in this special issue were solicited 
on the basis of the expertise of the authors and 
the relevance of their work to the field of 
health informatics. They were refereed by the 
special editors and the Editorial Board during 
the period | June to 1 July 2003. 

Peter Williams and Paul Huntington 
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In the UK, both anecdotal and research evidence points to 
ever greater and more sophisticated use of the Web to 
provide health information and advice. The study reported 
here adds to this research with an online survey of Internet 
users’ reported use of the Web to access information about 
health and their opinions about the advice that can be 
obtained there. Over a period of three weeks more than 
1,300 people responded to an online questionnaire 
produced by The British Life and Internet Project; 81 per 
cent or 997 of the respondents were British. The prime 
purpose of the questionnaire was to obtain information on 
the characteristics of the users of health information Web 
sites, to obtain feedback regarding for what they used 
online health sites and what were the perceived outcomes 
associated with using online health information. 
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Introduction 


The researchers have spent the last three years 
charting the first footfalls of the digital health 
consumer on behalf of the Department of 
Health (Nicholas et al., 2000-2003, 2001- 
2002). In this regard we have explored the use 
of a variety of digital platforms — touch-screen 
information kiosks, digital interactive 
television and the Internet, on a national and 
regional level. As part of these studies we have 
evaluated in detail individual consumer health 
Web sites SurgeryDoor and NHS Direct 
Online (Nicholas et al., 2002a,b), and are 
currently evaluating another, MedicDirect. 
However, we have not attempted to survey 
the use of the Web as a whole in a UR 
consumer health information context. 
Therefore this paper represents a first step to 
evaluating what UK health consumers are 
doing online, which sites they visit and what 
are the main outcomes of using online health 
information. With so much investment and 
hope being expended in digital health there is 
a crying need to discover what are the impacts 
or outcomes of using today’s digital 
information systems. 


Literature review 


It has been long established that health 
information is one of the most frequently 
sought topics on the Internet. Thus, Boseley 
(1999) estimated that “More than 40 per cent 
of all Internet users ... have sought health- 
related information, making it second only to 
pornography in popularity.” The consumer 
uptake of electronic health information has 
been truly phenomenal, and there is abundant 
evidence that this is the case, especially with 
regard to the USA. A Harris poll (Taylor, 
2000) found that 98 million adults in the 
USA had used the Web to find health 
information. This was up 81 per cent from 
two years earlier, during which time the 
number of Internet users who had never 
looked for health information had declined 
from 29 per cent of all users to 14 per cent. In 
a more recent study, PEW Internet & 
American Life Project (2003) found that 70 
million US adults had used the Web to get 
health or medical information (from looking 
for disease information to downloading fat- 
free recipes). 
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Reuters (2003) in an international study 
reported that on average 53 per cent of 
Americans used the Internet to search for 
health information. However, a survey by the 
Gale Group (2003) seems to be at odds with 
this. They found that 62 per cent of US adults 
did not seek health information from any 
source other than their doctor in the previous 
year and, further, that only 16 per cent of the 
population sought health information on the 
Internet. Of course, it is as well to remember 
that the online community is still a relatively 
small, but growing, subset of the total 
population. In the consumers’ thirst for health 
information they will even go to sites once 
thought to be the sole domain of health 
professionals. Thus an average of 220 million 
searches are performed annually on the 
National Library of Medicine’s Web site, of 
which an estimated one third are thought to 
be made by the general public (Wood et al., 
2000). It has been argued that “a driving force 
behind demand for online health information 
is the shortage of information easily obtained 
from traditional channels” (Miller and 
Reents, 1998). 

With regard to Europe, a recent EU 
Eurobarometer survey on online health 
information (European Commission, 2003) 
found that across the EU nearly one in four 
Europeans (23 per cent) uses the Internet to 
obtain health information. However, the 
picture varies considerably between countries: 
in Denmark and The Netherlands around 40 
per cent of people use the Internet for health 
information, while in Greece, Spain, Portugal 
and France usage is at 15 per cent or less. 
Health professionals, such as doctors and 
pharmacists, are still by far the most 
important source of health information for 
Europeans and the traditional media — 
television, newspapers, and magazines — still 
outperform the Internet. Other key findings of 
the survey are that medical and health 
organisations achieve the highest trust rating 

on-health issues (84 per cent), while 
` businesses and political parties receive the 
lowest (16 per cent and 11 per cent 
respectively). 

In a very recent study of over 1,100 UK 
Internet users, Brown and Williams (2003) 
reported that 90 per cent claimed to have 
visited a Web site for health information and/ 
or medical treatment in the past 12 months. 
Over 70 per cent of their respondents had 
reportedly visited a Web site for health or 
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medical information or advice at least ten 
times during the previous year. Nearly half 
(49 per cent) had reportedly visited the 
National Health Service Web site, while over 
one in five (22 per cent) said they had visited 
a pharmaceutical company Web site. 

There is clearly some debate about the 
effectiveness of online health information, 
although most researchers have come to the 
conclusion that it can be effective. However, 
on the negative side, Baker et al. (2003) in a 
survey of 4,764 people found that only 40 per 
cent had used the Internet and that 94 per 
cent of these said that the information found 
did not affect the number of times they visited 
a doctor. However, more positively, 
Kalichman et al. (2003) found evidence that 
Internet health-related information did have 
health benefits for people living with HIV/ 
Aids. Members of the present research group 
(Nicholas et al., 2001) have also carried out a 
survey on the use of information on the 
SurgeryDoor consumer health Web site 
(www.surgerydoor.co.uk) and found evidence 
of positive health outcomes as a result of using 
the service — in particular that 27 per cent of 
SurgeryDoor users (respondents) said they 
used the information instead of visiting the 
doctor. McKillen (2002) concluded that the 
Internet has had a positive effect on the 
doctor-patient relationship, a result also 
supported by Reuters (2003). 

A study by PEW (2000) found that most 
people looked for medical information about 
a specific condition rather than for 
information about healthy lifestyles or health- 
care services, and a great many sought 
information on behalf of family and friends as 
well as for themselves. A total of 48 per cent 
said that the advice they found on the Web 
improved the way they took care of 
themselves and 55 per cent claimed that the 
Internet had improved the way they accessed 
health information. The vast majority of those 
seeking information for themselves found the 
information they located useful and 
educational and nearly half said that it 
affected their decisions about diet and 
exercise, treatments and about future take-up 
of health-care services. However, most people 
do not use the Web instead of visiting a 
doctor. It appears the majority go online after 
they have seen a doctor to gather further 
intelligence, confirming the need for, and 
benefit of, information as an adjunct to the 
doctor-patient relationship. 
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Women were more likely than men to seek 
online health information. The survey found 
that 63 per cent of women with Internet 
access had sought health information, against 
46 per cent of men, with, on a typical day, 59 
per cent of searches being carried out by 
women. Women, and in particular mothers, 
frequently adopt the role of health carers for 
their children and other family members. 
Health information seekers are predominantly 
of middle years (between the ages of 30 and 
64) rather than from younger or older age 
groups. Not surprisingly, the longer someone 
has had Internet access, the more likely is it 
that they have looked for health information. 
A total of 42 per cent of users, once they have 
found a useful site, especially men, were likely 
to bookmark it for future reference. This still 
leaves a large proportion of health surfers. 

Finally, the Internet health organisation 
Health on the Net Foundation’s Evolution of 
Internet Use for Health Purposes (HoNF, 2001) 
survey found that the vast majority of Internet 
health information seekers searched for 
medical literature (83 per cent), with 67 per 
cent looking for the “description of a disease”, 
and 38 per cent for clinical trial information. 

Again in the UK, Brown and Williams 
(2003) reported that over nine out of ten of 
their respondents commented on the 
convenience of having immediate health 
information available via the Internet. Usage 
of the Internet for health was high across age 
groups and over other demographic groups 
and whether or not respondents indicated 
that they had recently experienced serious 
health problems themselves. Follow-up 
qualitative research indicated that Internet 
users surfed the Web for health information 
for general enquiries as well as when they 
suffered specific complaints or conditions. 
The same study revealed that, while seen as a 
useful resource, many survey respondents (70 
per cent) questioned the accuracy of 
information they received over the Internet, 
and slightly more (72 per cent) expressed 
concerns about not being able to validate 
whether the information was credible. A 
significant minority of Brown and Williams’ 
respondents (42 per cent) had other 
reservations about being able to interpret the 
information they found. 

Interactivity is an important aspect of new 
communications technologies. However, the 
willingness of people to use interactive media 
for health purposes depends upon the 
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application. Brown and Williams (2003) 
reported that most UK users of the Internet 
for health information they surveyed (82 per 
cent) said they would be willing to book an 
appointment with their doctor online, far 
fewer (33 per cent) were comfortable with the 
idea of dealing with their doctor remotely in 
this way and 29 per cent would be prepared tc 
provide personal medical details to a medical 
Web site that provided diagnostic tools. 

Hence, the Internet is seen as a valuable 
health information source, provided that the 
authority and credibility of the information 
can be assured or verified. It is also seen as 
having interactive potential, but more as an 
advanced information tool than as a 
diagnostic and treatment source. 


Aims/objectives 


This study sought to extend our knowledge 

concerning how UK residents used online 

health Web sites and to what ends. 

Specifically, the study aimed to discover: the 

proportion of people using health sites, 

information about the kinds of people that 

used online health Web sites, what particular 

sites they used, and the outcomes associated 

with using this online information. In 

particular the following information was 

sought about the British: 

* likely audience reach of online consumer 
health information; 

* gender, age and ethnicity variations in 
use; 

* the uses to which digital health 
information is put; 

* the type of online service used (i.e. online 
support groups, e-mail enquiry service); 

* the outcomes associated with using online 
health information; and 

+ the Internet sites that are used. 


A subsidiary aim was to highlight, where 
appropriate, the differences between UK and 
overseas Internet users. 


Methods 


An online questionnaire survey was carried 
out by The British Life and Internet 
Project[1], a research group, of which the 
authors are members, on a population of 
Internet users resident in the UK, yielding a 
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sample of 1,322 respondents. Respondents 
were asked to provide information about their 
Internet-related health search behaviour. 
Those who indicated that they had used the 
Internet to search for online health 
information were then asked further questions 
as to their use of health Web sites and the 
outcomes associated with using the Internet 
for health information. All respondents 
provided self-report data on their Internet 
access history, and frequency of the Internet 
at home and at work, and personal details on 
their demography (sex, age, ethnicity, marital 
status). They provided information about 
access to computer equipment at home and 
where this equipment was located. They were 
then presented with a list of health-related 
behaviour questions and were asked in each 
case whether they had ever or never used the 
Internet for these purposes. The list 
contained a variety of content retrieval, 
interactive and transactional activities. 
Respondents were asked to say whether using 
the Internet had changed the way they obtain 
health information. 

As with all online surveys, this one was 
dependent on a database of e-mail addresses. 
The British Life and Internet Project, of 
which this survey is a part, has compiled, with 
the help of The Independent newspaper, a 
database of 30,000 e-mail addresses. 
Members of this online universe were notified 
about the survey by e-mail containing a link to 
a Web-based online survey. It is not known if 
this sample is representative of the general 
UK population, although clearly The 
Independent attracts a particular broadsheet 
readership. Online respondents were self- 
selected volunteers. There was no 
opportunity to control respondent selection in 
advance to ensure randomness or to set 
quotas by key demographics. These caveats 
therefore need to be borne in mind when 
considering the data provided here. 


Characteristics of sample 

The data reported here were collected over a 
24-day period from 2-26 May 2003. A total of 
1,322 Internet users replied, giving a response 
rate of 4.4 per cent. However, responses 
comprised a wide demographic mix. In all, 59 
per cent of respondents were male and 41 per 
cent were female. Respondents covered a 
range of age groups: 18-24 (10.6 per cent); 
25-34 (19.0 per cent); 35-44 (24.0 per cent}; 
45-54 (24.3 per cent); 55-64 (15.2 per cent}; 
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65-74 (5.6 per cent); and 75+ (1.4 per cent). 
Just under onein three respondents was single 
(31.3 per cent), nearly half were married 
(47.0 per cent), one in six was living with a 
partner (13.6 per cent), while the remainder 
were either divorced/separated (6.3 per cent) 
or widowed (1.3 per cent). About one in four 
(23.4 per cent) had children in their 
household aged up to 16 years. Most 
respondents (90.2 per cent) were ethnically 
white and of European origin. Others 
variously identified themselves as African 
(1.7 per cent), Caribbean (1.0 per cent), Arab 
(0.5 per cent), Indian/Pakistani/Bangladeshi 
(1.4 per cent), Chinese (0.7 per cent), Other 
Asian (1.0 per cent), or Other (3.5 per cent). 

The characteristics of UK respondents were 
similar with regard to gender, 58 per cent 
being male and 42 per cent female, Though 
UK respondents were younger, 31 per cent 
were aged under 34 years. Fewer UK 
respondents were married and more were 
single: 46 per cent of UK respondents were 
married compared with 52 per cent of 
non-UK respondents. Of the respondents, 27 
per cent said that their household included 
children. In addition there were marginally 
more UK respondents in full-time 
employmen:, 66 per cent compared with 
62 per cent. 

A total of 81 per cent of respondents came 
from the UK. This study predominantly 
examines UK users, although some 
international comparisons are made where 
appropriate. In fact, there were substantial 
differences between UK and non-UK 
respondents that require further discussion. It 
was found, for example, that non-UK 
respondents were more likely to be males 
(64.5 per cent) than UK respondents (54 per 
cent)[2]. Furthermore, non-UK respondents 
were more likely to be older — 11 per cent 
were aged between 65 to 74 compared with 
5 per cent of UK 65 to 74-year olds([3]. 
Interestingly, previous literature has identified 
a high level of Internet use by US seniors 
(Gale Group, 2003). In addition non-UK 
users were more likely to be married: 54 per 
cent compared with 47 per cent of 
respondents from the UK were married[4]. 
Lastly, non-UK users were much more likely 
to report that they were in excellent health: 
37.1 per cent confirmed this compared with 
27 per cent of UK respondents[5]. 

A potential, but not surprising, bias in the 
sample should be noted. More respondents 
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than expected said that they read broadsheet 
rather than tabloid papers. While respectively 
27.5 per cent and 32.0 per cent of UK 
respondents identifed themselves as 
Independent and Guardian readers, only 4.5 
per cent of respondents said that they read 
The Sun newspaper and 5.4 per cent the 
Daily Mirror. 


Results 


Internet use 

A total of 66 per cent of UK respondents 
accessed the Internet from home and 28 per 
cent from work; the remainder used a 
combination of work and home. 
Furthermore, 18 per cent had accessed the 
Internet within the last week, 50 per cent 
within the last month and 85 per cent in the 
last six months. Most UK respondents (49 
per cent) accessed health information for 
themselves, 16 per cent for someone else, 10 
per cent for another relative, 7 per cent for a 
parent, 7 per cent for a child and 7 per cent 
were just browsing. 


Reach 

Reach tells us the percentage of individuals 
who availed themselves of the service who 
potentially could have had accessed it (the 
population of subscribers). Reach indicates 
how well used a resource is in a particular 
community. This survey asked Internet users 
if they had ever used the Internet to look for 
health-related information. In all 80.5 per 
cent of respondents had used the Internet for 
this purpose. The percentage for UK only 
users was much the same at 80.3 per cent. 
However, this is likely to be an over- 
estimation, as the questionnaire was self- 
selecting and we would expect users of health 
information to be more willing to respond to 
the questionnaire. 

The estimated reach figure was higher for 
women than for men: 88 per cent of women 
with Internet access had used it to find health 
information, whereas this was only true of 75 
per cent of men[6]. Furthermore, younger 
users were less likely to use the Internet in this 
way: 70 per cent of those aged between 18 to 
24 had done so compared with about 84 per 
cent of those aged 35 and over[7]. This could 
be down to the fact that young people are 
generally healthier or that they do not have 
the facilities to search the Web so regularly. 


Volume 55 - Number 5/6 - 2003 - 261-276 


Those living with a partner were found to be 
more likely to use the Internet to search for 
health information: 91 per cent said that they 
did so compared with 83 per cent of married 
respondents[8]. There was no significant 
difference between Caucasian and non- 
Caucasian respondents. 


What do people do when they go online? 
Clearly what people do online is important, as 
this gives us an idea of why people are 
searching the Internet. Respondents were 
given a list of things people were thought to 
do when online and asked to indicate if this is 
something they had done or not done. The 
results are given in Figure 1. 

Almost all of the UK respondents (97 per 
cent) who had accessed the Internet for health 
information had done so to look up 
information about a particular illness and 
condition. In total 57 per cent had gathered 
information in respect of a visit to the doctor 
and 52 per cent had used the Web to look for 
information or advice about nutrition, 
exercise, or health weight control. Just under 
half of UK respondents had looked for 
information about alternative medicines; this 
must be regarded as a surprisingly high figure. 
Furthermore, 44 per cent had looked for 
information about a sensitive health topic that 
is difficult to talk about. This is significant in 
that many of these respondents would be 
unlikely to talk to a medical professional 
about these issues. A total of 40 per cent 
looked for information about a mental health 
issue like depression or anxiety and 34 per 
cent looked for information about innovative 
or experimental treatments. Only 23 per cent 
of respondents actually sought to diagnose or 
treat a medical condition. 

Surprisingly, perhaps, more women said 
that they had accessed mental health pages: 
49 per cent did so compared with 33 per cent 
of men[9]. Women were also more likely to 
access alternative remedy information: 57 per 
cent had done so compared with 41 per cent 
of men[10]. Women were more interested in 
accessing pages on nutrition, exercise and 
weight control: 62 per cent did so compared 
with 43 per cent of men{11]. 

There were large age differences too. Again, 
surprisingly, as those aged between 25 and 34 
were more interested in mental health 
matters: 52 per cent had done so compared 
with 41 per cent of respondents in the 35 to 
44 age group. Significantly, younger 
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Figure 1 Health information sought-when online 
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respondents were more likely to be interested 
in using the Internet to access information 
related to a sensitive topic: 62 per cent of 
those aged under 34 had done so compared 
with 40 per cent aged 35 and over. Younger 
respondents were more likely to access 
information on nutrition, exercise and weight 
control: 64 per cent of those aged 34 and 
under had done so compared with about 

50 per cent of those aged 35 and over. Those 
over 35 were more likely to use the Internet to 
look for information on prescription drugs: 60 
per cent did so compared with about 45 per 
cent of younger respondents. 

Non-Caucasian respondents were more 
likely to say that they had used the Net to look 
at mental health pages: 53 per cent did so 
compared with 39 per cent of Caucasian 
respondents[12]. They were also more likely 
to search for a sensitive topic: 60 per cent of 
non-Caucasians did so compared -with 43 per 
cent of Caucasians[13]. In addition, non- 
Caucasian respondents were more interested 
in information leading to self-diagnosis: 

35.1 per cent had done this compared with 
22 per cent of Caucasian respondents[14}]. 

Respondents searching for themselves were 
more likely to say that they had accessed a 
sensitive topic: 50 per cent did so compared 
with 34 per cent for those who did not{15], 
and regarding access to nutrition exercise and 
weight control pages 58 per cent compared 
with 42 per cent[16]. However, these 
people were less likely to laok for new 


treatments: 31 per cent compared with 39 per 
cent[17] ard less likely to access pages on 
prescription drugs: 51 per cent compared 
with 60 per cent{18]. 

Those people who rated themselves as 
being in poor or fair health were more likely to 
say that they had accessed mental health 
pages: 68 per cent and 52 per cent, 
respectively, said that they did so compared 
with 36 per cent of those who rated their 
health as excellent[19] Additionally, those in 
poor or fair health were more likely to look for 
information regarding a doctor’s visit: 
respectively 74 per cent and 66 per cent did so 
compared with 49 per cent who said that they 
were in excellent health. Finally, those in poor 
or only fair health were more likely to have 
accessed prescription drug information: 83 
per cent and 64 per cent respectively had 
done so compared with 46 per cent of people 
in excellent health[20]. 

There were some variations in behaviour 
according to the newspapers people said they 
read, Those reading The Independent were less 
likely to have looked at sensitive health 
information: 39.4 per cent compared with 47 
per cent[21] had done so, and these readers 
were also less likely to have accessed 
alternative remedy information: 43 per cent 
compared with 51 per cent of non-Independent 
readers[22]. Guardian readers, however, were 
the reverse and were more likely to have 
accessed sensitive health topics: 51 per cent 
compared with 41 per cent had done so[23], 
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and were more likely to have looked for 
alternative remedies: 53.6 per cent had done 
so compared with 45.2 per cent[24]. 
Guardian readers were more likely to search 
for a doctor or hospital: 37 per cent compared 
with 28 per cent of non-Guardian readers[25]. 
There was suggestive evidence that Sun 
readers were more willing to access pages 
related to nutrition, exercise and weight 
control. 

Married people were the least interested in 
nutrition, exercise or weight control: 46.6 per 
cent compared with 58 per cent of other 
groupings[26], and were least interested in 
looking at information on a sensitive topic: 
36 per cent had done so compared with 50 
per cent for non-married people[27]. There 
was suggestive evidence that full-time 
housewives were more interested in 
prescription drugs (72 per cent); while those 
in full-time education were least interested 
(40 per cent) [28]. 

Respondents were further asked about the 
online services and functions that they used 
(Figure 2). 

Nearly half of all UK respondents looked to 
see what company or organisation was 
responsible for providing the advice or 
information that appeared on the health Web 
site. This indicates, surprisingly, that half did 
not and did not seem to be too bothered 
about the source of their information. Only 
just over a third of respondents (38 per cent) 
kept a health site “bookmarked”, or saved as a 


Figure 2 Online health services or functions used 
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“favourite place”. This argues that health 
users are quite promiscuous: happy, for 
instance, to have a Web search engine make 
their choice for them. A total of 15 per cent 
bought medicine or vitamins online and only 
15 per cent had clicked on a health or medical 
Web site’s privacy policy to read about how 
the site uses personal information supplied to 
it. Of the respondents, 14 per cent 
participated in an online support group, 

10 per cent used e-mail or had gone to a Web 
site to communicate with a doctor or doctor’s 
office, while about 8 per cent actually 
described a medical condition or problem 
online in order to get advice from an online 
doctor. 

This contrasts sharply with the expected or 
forecast take-up of specific online interactive 
services. Reuters (2003) reported that 45 per 
cent of Europeans would “like” to e-mail their 
doctor. While many people have indicated a 
willingness to make remote appointments 
bookings with their doctors, however, far 
fewer, as yet, are comfortable with remote 
diagnosis (Brown and Williams, 2003). The 
difference between expected and actual 
take-up might partly reflect the fact that many 
doctors and clinics do not currently offer 
patients e-mail contact. If this is the case, we 
would expect the percentage take-up figure to 
increase (as surveys are repeated), as more 
medical outlets offer e-mail or online 
facilities. Other reasons for the low take-up 
might be the unwillingness of users to 
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communicate online with their doctor from a 
work situation where their actions can be 
viewed or monitored. 

Older respondents were more likely to have 
bought medicine and vitamins online: 42 per 
cent of those aged 65 to 74 had done so 
compared with 14 per cent of those aged 
between 35 to 44[29]. There was suggestive 
evidence that those aged between 35 to 44 
were more likely to join an online support 
group — 20 per cent of this age group had 
done so compared with 15 per cent of those 
aged between 45 and 50 and 9 per cent of 
those aged 25 to 34[30]. Those aged 34 and 
under were less likely to use a bookmark: 
about 34 per cent did so compared with 
40 per cent of those 35 and over[31]. Older 
respondents were more likely to check the 
source. About 55 per cent of those aged 35 
and over did so compared with 40 per cent 
aged between 25 and 34 and 25 per cent aged 
between 18 to 24[31]. 

Full-time housewives were more likely to 
buy online medicine and vitamins: 28 per cent 
had done so compared with 12 per cent in 
full-time employment[32]. There were no 
data to suggest that this was related in any 
way to having children. This group was also 
more likely to join online support groups: 44 
per cent said that they had done so compared 
with only 13 per cent who were in full-time 
employment. Furthermore, full-time 
housewives were more likely to seek the 
advice of an online doctor: 22 per cent did so 
compared with 6 per cent in full-time 
employment[34]. Search disclosure (Nicholas 
et al., 2003) describes the users’ unwillingness 
to access sensitive information in locations 
where they may be observed by other people 
and this may account for the low take-up of 
online services by this group. Future research 
will clarify this. 

Those people working from home were 
more likely to read the online privacy 
statement: 17 per cent said that they did so 
compared with 10 per cent of those who 
accessed from work. Perhaps, they just had 
more time. Further, those searching for 
information on a child’s health were more 
likely to read a privacy statement: 26.4 per 
cent had done so compared with 14 per cent 
of those not searching for a child(35]. Those 
searching for themselves were marginally less 
likely to check the online source: only 
46 per cent did so compared with 
53 per cent[36]. 
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Those people in poor health were more 
likely to bookmark a site: 58 per cent in poor 
health and 53 per cent of those who had fair 
health did so compared with 36 per cent of 
those in excellent health[37]. Those in poor 
health were also more likely to join an online 
support group: 60 per cent had done so 
compared with 15 per cent of those with only 
fair health and 11 per cent in excellent 
health[38] — suggesting, of course, that poor 
health is a driver for using support group 
facilities. 


Outcomes 

What are the benefits and outcomes of using 
online health information? The current survey 
indicated that there are many, with key 
applications shown in Figure 3. 

A total of 93 per cent said that the 
information found had helped in 
understanding more about an illness or injury 
and a relatively high 57 per cent of 
respondents said that the information found 
was sufficient for them to act upon to improve 
their health. A total of 58 per cent said that 
information found enabled them to help 
someone else, while 51 per cent said that it 
gave them information that the doctor had 
not given them. Of the respondents, 46 per 
cent said that the information found had 
confirmed what the doctor had told them, 
while 38 per cent said the information found 
had given reassurance about recovery from an 
illness or injury. A total of 26 per cent said 
that the information found had affected their 
decision about whether to see a doctor. 

In respect of the doctor, 38 per cent said 
that the information found had helped them 
ask new questions of the doctor, while 5 per 
cent said that they sought a second opinion 
based on the information found. 

Older users said that the information found 
had given them more information than the 
doctor provided: 61 per cent of those aged 
between 45 and 54 said this compared with 
31 per cent aged 18 to 24[39] This was also 
true for those in poor health: 74 per cent of 
those in poor health and 63 per cent of those 
in fair health said that the information found 
was more than that provided by the doctor. 
This was only true for 40 per cent of those in 
excellent health[40]. In addition those 
married or living with a partner were also 
likely to find more information on the 
Internet than that provided to them by their 
doctor: about 57 per cent of these two groups 
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Figure 3 Perceived benefits of Internet health information 
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said this compared with 43 per cent of single 
people. 

Women were more likely to find 
information that helped someone else: 60 per 
cent said they had done so compared with 
53 per cent of men[41]. This was also true of 
those respondents with children: 63 per cent 
said this compared with 53 per cent[42]. 
Older respondents also were found to be more 
likely to find information that helped 
someone else. About 60 per cent of those 
aged over 35 said this, compared with 46 per 
cent of those aged between 25 to 35 and 37 
per cent of those aged 18 to 24[43]. This was 
further true of those in a relationship. A total 
of 58 per cent of those married or living with a 
partner had found information that helped 
someone else compared with 48 per cent of 
single users and 71 per cent of widowed, 
separated or divorced respondents [44]. 

Respondents in fair health felt most 
reassured by information found — 46 per cent 
felt reassured. Those in poor health felt the 
least reassured: only 11 per cent of this group 
felt reassured by the information found[45]. 
Both those in poor and in fair health said that 
the information found had confirmed what 
the doctor said. About 58 per cent of this 
group said the information found had done 
this compared with 49 per cent of those in 
good health and 33 per cent of those in 
excellent health[46]. 

There was suggestive evidence that non- 
Caucasians were more likely to be reassured 








by the information found: 50 per cent were 
reassured compared with 36 per cent of 
Caucasian respondents[47]. 

Respondents were also asked if obtaining 
health and medical information on the 
Internet influenced them in changing their 
health-related behaviour. Figure 4 provides 
the results and appears to show that the Web 
has a beneficial impact. 

Nearly half of the respondents (45 per cent) 
said that information found had caused them 
to think about the things they eat. A total of 
40 per cent said that information found had 
made them more aware of the need to live a 
healthy life, while 38 per cent had said it 
encouraged them to take more exercise. Just 
under a third of users said that the 
information found led them to eat more fruit 
and vegetables, while 26 per cent said that it 
had encouraged them to relax more. 
Interestingly, Internet information seems to 
have the least impact on smoking and 
drinking habits. Only 4 per cent of 
respondents were influenced to give up 
smoking, while 11 per cent said that the 
information found had influenced the amount 
of alcohol consumed ~ probably showing how 
addictive are tobacco and alcohol. Six per 
cent said that the information found had 
caused them to go for regular health checks 
with their doctor. 

There was evidence that the figures given 
above were even higher for those respondents 
outside the UK. For example, 65 per cent of 
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Figure 4 Behaviour that online information has changed 
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non-UK respondents said that they had found 
information that resulted in a change in eating 
habits. The outcome percentages for non-UK 
respondents were also higher for giving up 
smoking (9.7 per cent), taking vitamins (39.8 
per cent), eating fruit and vegetables (54.1 per 
cent), exercise (57.1 per cent), relaxation (47 
per cent), alcohol (18 per cent), check-ups (19 
per cent) and a healthy lifestyle (60 per cent). It 
is not thought that UK respondents were 
necessarily poorer in taking up online 
information and these differences may reflect a 
sampling bias in non-UK Internet health users. 

Women were more likely to have found 
information that led them to a change in diet: 
50 per cent of women said so compared with 
42 per cent of men[48]. Older respondents 
appeared more open to encouragement to 
take vitamins and supplements: 27 per cent of 
those aged 55 and over were encouraged to do 
so compared with 17 per cent of those aged 
25 to 54[49]. Older and younger respondents 
said that the information found had helped 
them take more exercise: this was so for 53 
per cent of those aged 18 to 24 and 44 per 
cent of those aged 55 to 74 compared with 32 
per cent of those aged 35 to 54[50] . 

There was some evidence that full-time 
housewives had found information that led to 
a change in eating habits, and this was true of 
72 per cent of this group[51]}. This group 
were also more likely to have found 
information that encouraged more exercise 


(67 per cent)(52], and relaxation (56 per 
cent) [53]. 

Those searching for themselves were more 
likely to say that the information found had 
resulted in them changing their behaviour; 
this was true for eating habits: (51 per cent 
said so), exercise (44 per cent), vitamins and 
supplements (21 per cent), relaxation (28 per 
cent) and a healthy lifestyle (45 per cent). 
Typically, the relationship is characterised as 
follows: 51 per cent of those searching for 
themselves had changed their eating habits 
compared with 38 per cent had done so but 
who did not say that they were searching for 
themselves[54]. 

Those in poor or fair health were more 
likely to say that the information found had 
encouraged them to relax more: 55 per cent of 
those in poor health and 32 per cent in fair 
health said so compared with only 21 per cent 
of those in excellent health[55]. 

There was suggestive evidence that Sun 
readers were more likely to have found 
information that resulted in them giving up 
smoking and drinking less alcohol. Of this 
group, 11 per cent had given up smoking 
compared with 4 per cent of non-Sun 
readers[56] and 23 per cent said information 
found had influenced the amount of alcohol 
consumed [57]. Suggestive evidence further 
indicates that these readers might be more 
likely to report that information found had led 
to a healthier lifestyle: 56 per cent of Sun 
readers said so compared with 39 per cent of 
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non-Sun readers[58]. However, the sample is 
not representative of The Sun newspaper 
readers. There were only 36 respondents who 
were Sun readers and 15 of those also read 
The Guardian — further research is needed to 
clarify these findings. 

Independent readers were less likely to find 
information that resulted in a change of eating 
habits: 41 per cent said so compared with 49 
per cent of non-Independent readers[59]. 

Respondents were also asked if they ever 
went online instead of visiting a doctor or 
clinic: 28.5 per cent of respondents said that 
they had, but only once or twice. This figure 
is similar to that found in a study by Nicholas 
et al. (2001), where 27 per cent of users 
(respondents) of an online health site 
(www.surgerydoor.co.uk) said they had used 
online information found instead of visiting 
the doctor. With regard to the current study, a 
multivariate model found that those who used 
the Internet for diagnosis were nine times 
more likely to use online health information in 
this way compared with respondents who did 
not use the Internet for diagnosis[60]. In 
addition respondents who used the Internet 
to look up sensitive health information online 
were just under two and half times more likely 
to go online instead of visiting the doctor[61]. 
This seems to indicate that some users are 
doing this because they are actively avoiding a 
visit to the doctor. Further, that users will use 
online information found as a substitute for 
visiting the doctor and that people who do 
this have particular characteristics, needs and 
fears and online services need to cater for this 
type of online user and surrounding issues of 
this behaviour. 


Health sites used 

People were asked about the sites they visited. 
Figure 5 gives the sites that have ever been 
used by UK respondents. We should bear in 
mind that a previous analysis indicated that a 
lot of people never bookmarked their sites, so 
there is an issue of how accurately 
respondents might actually remember the 
sites they have visited. 

The most accessed site, by some margin, 
was NHS Direct Online. Nearly two-thirds of 
all UK respondents had visited this site. The 
next most visited sites were Netdoctor 
(25 per cent) and Yahoo! Health (24 per 
cent). 

Figure 5 indicates that 64 per cent of UK 
respondents who had used the Net to look for 
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health information had used NHS Direct. 
This result cannot be generalised to the UK 
population of online users’ search for health 
information. This is because the sample is 
heavily biased towards a Guardian newspaper 
readership — 33 per cent of the sample read 
this paper. The sample did not include many 
Sun readers (4.5 per cent), or Daily Mail 
readers (5.7 per cent). This is important as 
Guardian readers were found to be just under 
twice as likely to use the NHS Direct site, 
while Sun readers were half as likely to use the 
site[62]. That is, compared with Sun readers, 
Guardian readers were about four times more 
likely to use NHS Direct. Clearly, the 
performance of the NHS Direct site here 
significantly benefits from the skewed sample 
towards a Guardian readership. 

The high use recorded for NHS Direct in 
part reflects the sample bias towards Guardian 
readers. This bias will also impact on the 
performance of other sites. For example, in a 
recent survey of the SurgeryDoor site 
(Nicholas and Huntington, 2003) it was 
found that 21 per cent of respondents were 
Daily Mail readers, 12 per cent were Sun 
readers and only 9.9 per cent were 
Guardian readers. Clearly, the bias towards 
Guardian readers for the current study will 
weigh against a site like SurgeryDoor. 
Huntington et al. (2003) argued that different 
Web sites will attract different types of socio- 
economic user groups. They argue that no 
one site will appeal to all user groups, and that 
language, presentation and design differences 
between sites will mean that sites will attract 
different user groups. One way of describing 
user group differences is by the paper they 
normally read. The NHS site appears to 
attract more of a broadsheet — and less of a 
tabloid-type user. 

Most people (53 per cent) visited between 
two to five of the listed sites, just over a third 
of respondents (35 per cent) had visited just 
one site, 10 per cent had visited between 6 to 
10 per cent of the sites and 4 per cent had 
visited 11 or more of the listed sites. 


Returnees 

Coming back to a service constitutes 
conscious and directed use. People might 
arrive at a site by accident — and, of course, 
that constitutes “use”, but they are unlikely to 
arrive at the same site again by accident. All 
this makes return visits a powerful 
performance indicator. The new media 
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Figure 5 Health Web sites visited 
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industry has referred to this phenomenon as 
Web site “stickiness”. A site with a high 
percentage of returnees can be regarded as 
having a “brand” or loyal following. All this 
makes return visitors a powerful performance 
indicator. 

It seems likely that the more times a user 
returns to the site, the more likely they are to 
be satisfied with the site — and this is indeed 
what Morris (1997) found. The idea of 
satisfaction is linked to the likelihood of a user 
returning to the site. Thus, if a user makes 
one visit to a Web site and does not return, 
then it might be argued that the user has 
voted with their feet. Figure 6 shows the 
users’ repeat behaviour over five selected sites 
visited. 

In terms of repeat use Yahoo! Health 
performs poorly and recorded the least 
proportion (24 per cent) of users willing to 
come back three or more times to the site. 
SurgeryDoor, on the other hand, performed 
particularly well by this metric — about half its 
users returned to the site three or more times 
— obviously liking what they saw. 

These figures are broadly matched by 
ratings of how useful was the information 
found in each site. Figure 7 shows how useful 
respondents found the information by sites 
visited. Of the respondents, 88 per cent who 
had used the SurgeryDoor site said that 
information found was either always or mostly 
useful, but this was only true for 73 per cent 


Figure 6 Number of times visited five Web sites 
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of respondents who had used the Yahoo! 
Health site. 

It was decided to look more closely at the 
NHS Direct site by looking at respondents 
who had used the site according to whether the 
information found had influenced or changed 
their behaviour. The question asking whether 
information found had changed behaviour 
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Figure 7 Usefulness of information found by sites visited 
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Direct had found information that had 
encouraged them to change their eating 
habits, this being more true, the greater the 
number of visits to the site, except those 
visiting ten or more times. Further, 
respondents who had used NHS Direct had 
increased their consumption of fruit and 
vegetables (but only for those visiting the site 
three to nine times). These respondents were 
also more likely to have increased their visits 
to the doctor; however, this again was only 
true for respondents visiting the site ten or 
more times. NHS Direct users (respondents) 
were also more likely to adopt a healthy 
lifestyle. This was more true, the greater 
number of times the user visited the site. 

It was thought that the greater the number 
of sites visited, the greater the likelihood of a 
healthy behaviour outcome. This is based on 
an information model that argues that not all 
sites will present the information in the same 
way, detail or in the same design. Users 
“benefit” from collecting information from a 
number of sites, partly because they find the 
information easier to digest on some sites than 
others, and partly because the process of 
jumping from site to site means that users can 
compare and contrast information, and this 
says something about how users become 
knowledgeable. To look at this in more detail 
it was decided to look at outcome as 
estimated by a change of behaviour by the 
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> followed the question related to sites used. The 
result for NHS Direct is given in Figure 8. 
Of the nine health options, NHS Direct had 
an impact on the behaviour with regard to 
four: eating habits, fruit and vegetables, 
having regular doctor check-ups, and having a 
healthy lifestyle. Those who had used NHS 
Figure 8 Perceived impact of Internet information on health-related behaviour by visits to NHS Direct 
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number of sites visited; the results are given in 
Figure 9. 

Out of the nine options impacting on health 
behaviour those that had visited more sites 
had a healthier outcome on seven of them. 
There was no outcome on smoking and 
drinking habits. The impact on some 
outcomes is considerable; for example, over 
90 per cent of those visiting ten or more sites 
said that the information found had changed 
their eating habits. There does appear to be 
evidence linking the number of sites a user 
visits with health outcomes. 


Conclusions 


The vast majority of respondents (80.3 per 
cent) had used the Internet to look up health 
information. This figure was higher for 
women than for men. Furthermore, younger 
users were less likely to use the Internet in this 
way, partly because they had a lesser health 
need and partly because they were not as 
Web-familiar as the more practised users. 
Clearly, people were attempting to get some 
of the information for which one might have 
originally gone to the doctor. Thus, 
97 per cent who had accessed the Internet for 
health information had done so to look up 
information about a particular illness or 
condition. They were also using this 
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information to enhance the patient-doctor 
relationship: 57 per cent were searching for 
information regarding a visit to the doctor; 52 
per cent were looking for nutrition, exercise, 
or weight control information. 

Respondents searching for themselves were 
more likely to say that they had accessed a 
sensitive topic: 50 per cent compared with 
34 per cent. Non-Caucasian respondents 
were more interested in information leading 
to self-diagnosis: 35.1 per cent had done this 
compared with 22 per cent of Caucasian 
respondents. 

Only about half of UK respondents 
(49 per cent) looked to see what company or 
organisation was providing the advice or 
information that appears on a health Web site. 

In terms of online services used, 14 per cent 
of respondents participated in an online 
support group, 10 per cent used e-mail or had 
gone to a Web site to communicate with a 
doctor or doctor’s office. About 8 per cent of 
UK respondents actually described a medical 
condition or problem online in order to get 
advice from an online doctor. Those 
respondents aged between 35 to 44, 
housewives and those in poor health were 
more likely to join an online support group. 

A total of 93 per cent said that the 
information found had helped in 
understanding more about an illness or injury. 
Further, a relatively high 57 per cent of 
respondents said that the information found 


Figure 9 Perceived impact of Internet information on health-related behaviour by number of different sites visited 
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was sufficient for them to act upon to improve 
their health, while 57 per cent said that they 
had found information that helped someone 

else. 

Nearly half of the respondents (45 per cent) 
said that information found had caused them 
to think about the things they eat. Further, 40 
per cent said that information found had 
made them more aware of the need to live a 
healthy life, while 38 per cent had said it 
encouraged them to take more exercise. 
Those in poor or only fair health were more 
likely to say that the information found had 
encouraged them to relax more. There was 
suggestive evidence that Sun readers were 
more likely to have found information that 
resulted in them giving up smoking and 
drinking less alcohol. 

Well over a quarter (28.5 per cent) of 
respondents said that they had gone online 
instead of visiting a doctor or clinic. 
Respondents who went online for a diagnosis 
or who used the Net to look up sensitive 
health information were also more likely to go 


online instead of visiting a doctor or clinic. 
\ 


Notes 


1 The British Life and Internet Project is operated by a 
consortium of researchers from the University of 
Sheffield, City University, eDigitalResearch.com, and 
The Independent Newspaper Group. 

2 6.81 df 0.009 

3 22.6 6df 0.001 

4 9.5 3df 0.023 

5 8.4 3df 0.038 

6 24 1df 0.000 

7 13.2 6df 0.04 

8 25.5 3df 0.000 

9 20.2 1df 0.000 

10 20.8 1df 0.000 

11 26.6 1df 0.000 

12 3.9 1df 0.049 

13 5.7 1df 0.017 

14 5.5 1df 0.019 

15 19.4 1df p = 0.000 

16 19.4 1df p = 0.000 

17 4.6 1df p = 0.032 

18 7.0 1df p = 0.008 

19 14.5 3df p = 0.002 

20 16.1 3df 0.001 l 

21 4.3 1df 0.038 

22 5.6 1df 0.018 

23 6.8 1df 0.009 

24 5.0 1df 0.025 

25 7.3 1df 0,007 

26 8.9 3df 0.030 

27 19.4 3df 0.000 

28 14.7 4df 0.005 

29 35.5 6df 0.000 
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30 12.3 6df 0.056 

31 18.2 6df 0.006 

32 33.9 6df 0.000 

33 9.9 4df 0.043 

34 19.2 4df 0.001 

35 7.5 1df 0.006 

36 4 1df 0.045 

37 14.2 3df 0.003 

38 32.5 3df 0.000 

39 20.0 6df 0.003 

40 20.6 3df 0.000 

41 3.7 1df 0.054 

42 5.2 1df 0.023 

43 22.7 6df 0.001 

44 11.1 3df 0.011 

45 10.9 3df 0.012 

46 21.6 3df 0.000 

47 4 1df 0.045 Fischer’s = 0.06 

48 5.6 idf 0.018 

49 11.8 6df 0.068 

50 15 6df 0.02 

51 9.2 4df 0.057 

52 13.4 4df 0.01 

53 13.1 4df 0.011 

54 11.4 1df p = 0.001 

55 12.2 3df 0.007 

56 Fischer's p = 0.067 

57 Fischer's p = 0.054 

58 Fischer's p = 0.081 

59 4.4 idf 0.037 

60 B = 2.217 Exp(B) = 9.17 Wald = 111.1 1df 

61 B= 0.867 Exp(B) = 2.37 Wald = 20.4 1df p = 0.000 

62 Guardian Exp(B) = 1.76** SE = 0.17 
Sun Exp(B) = 0.51t SE = 0.36 
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Introduction 


There has been much investment in providing 
consumers with health information on the 
Web. Indeed, the UK Government has 
proved to be a major investor with its NHS 
Direct Online Web site. The rapidly 
expanding NHS site and its numerous 
commercial and non-commercial rivals have 
proved exceedingly popular, if the number of 
“hits” they attract are anything to go by. 
Thus, in the case of the SurgeryDoor site over 
the period October 2001 to September 2002 
3,680,453 pages were viewed (excluding 
declared robots). As part of a three-year 
investigation (Nicholas ez al., n.d.) on behalf 
of the Department of Health we have been 
evaluating the use and impact of a number of 
UK sites, including NHS Direct Online and 
the commercial SurgeryDoor Web site, 
although in fact many were “promiscuous” 
users visiting a number of other health Web 
sites — about which we also sought data. 

The prime purpose of this particular study 
is to go beyond the headline hit figures and to 
identify and profile meaningful groups of 
health users, using a variety of criteria. This 
should help Web site managers to target their 
services more effectively, something that is 
beginning to concern them as they move 
towards providing a new range of personalised 
services. Without such an analysis there is a 
real danger that site owners (and consumers) 
will find themselves lost in the digital 
information fog. Site owners need to move on 
from hit counting and start obtaining 
meaningful user data. Most importantly, they 
need these data to help in menu and content 
construction, in that menus and content can 
be structured around identified information 
need groups. Menu and content structure 
could then reflect actual user interest rather 
than the producers’ or the site designers’ 
interest. Further, knowing more about user 
profiles and their site use leads to questions 
about a site’s information provision, 
presentation and also to a classification of 
sites. This can be thought of as working 
towards a classification that we are happy to 
apply in traditional media, for example, when 
referring to broadsheet and tabloid papers. 
The study described here constitutes a step 
towards such classifications. 
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Aims and objectives 


The purpose of the paper is to identify and 
distinguish meaningful groups of Internet 
health consumers. This is necessary because 
there are so many consumers (millions, 
probably) searching on such a range of health 
matters that generic health sites can never 
hope to meet all their needs as they are 
currently configured. Using feedback 
obtained via an online questionnaire mounted 
on a leading consumer health Web site, 
SurgeryDoor, and employing a range of 
statistical measures, we sought to examine the 
characteristics of the respondents and then 
categorise them according to these 
characteristics. 


Literature review 


Little work appears to have been undertaken 
in exploring the characteristics of Internet 
health seekers — the literature is, rather, 
preoccupied with issues of quality (e.g. 
Impicciatore et al., 1997) or accessibility (e.g. 
Eysenbach and Köhler, 2002). A wealth of 
literature has accumulated on the 
classification of Web use health information 
needs, although they have largely been 
concerned with the categorisation of Web 
documents (Norbert et al., 1999; Oates and 
Gibson, 2003). Health on the Net 
Foundation (HoNF) (1999) and the Pew 
Foundation (2000) have carried out a number 
of Internet surveys of health information 
consumers. However, these are reports on 
monitoring the basic statistics of the Internet 
health community and users over time by age 
and gender. 

For example, the Pew Internet & American 
Life Project did identify “health seekers” (55 
per cent of the Internet-user population) 
(Pew Foundation, 2000). Among the health 
seekers, 29 per cent go online weekly for 
medical information and 30 per cent monthly, 
while HoNF has found that a “steady advance 
of female users of Web-based health-care 
information” no longer applies (HoNF, 
1999), 

In an early non-digital study Pinder (1990) 
developed a typology of health information 
seekers on the basis of how sufferers and 
carers coped with the onset and development 
of Parkinson’s Disease. Three categories of 


user were identified — “seekers”, “weavers”, 
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and “avoiders”. The former sought as much 
information as possible, and used it as a 
central weapon in coping with the disease. 
Weavers sometimes sought information, and 
were selective about what they took on board. 
Avoiders believed that “the anxiety of not 
knowing was preferable to having their ... 
fears about the condition confirmed”. Leydon 
et al, (2000) in their study of cancer patients 
also found some reluctance to seek 
information. Huntington et al. (2002) in a 
study of text-based health information 
broadcast over a digital interactive television 
project identified four user types: active 
traditional media users, passive traditional 
media users, electronic passive/isolated users, 
and electronic sociable users. In particular, 
the paper identified new user types willing to 
explore health information via a DiTV 
platform. These users had not previously 
made use of this type of information. The 
authors raised the question as to the extent to 
which health information providers could use 
identified user groups to target specific users. 

Cyber Dialogue (2000) found that 
approximately half of all Internet-using health 
information seekers advised a family member 
or friend to see a doctor, changed their 
exercise or eating habits or made a “positive” 
decision related to their health treatment. A 
rather mixed bag of information need is 
shown in these results, with seekers looking 
for information on behalf of others, to 
improve their general health or to decide on 
the next steps they should take with regard to 
a current condition. Many others joined an 
illness support group after visiting a disease- 
specific Web site. Some work has been carried 
out on the impact of the Internet on such 
specialist online groups. Gann (1998) reports, 
for example, that participation in these 
forums focused around peer support, sharing 
of information on treatment advances and 
clinical trials. 

As with much else on the Web, it is often 
the marketers and consultancies which are 
leading the way in usage research, albeit in a 
fairly superficial way and with commercial 
rather than altruistic or academic ends. The 
Internet generation has been identified as 
being critical, demanding and sophisticated 
(Modalis, 2001). Further, marketers have 
categorised Internet users into economic 
groupings, such as the e-fluential “online 
movers and shakers” (Burson-Marsteller, 
2000), while Jepsen and Dipnall (2003) 
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looked at the classification of Internet use on 
ethical lines. One interesting study looking at 
online health information seeking has been 
that carried out by the Boston Consulting 
Group (BCG, 2001; Poensgen and Larsson, 
2001) with European health consumers. 
Their researchers found that users tend to 
have a focused and deep interest in 
information only about their specific 
condition or disease. They do not regularly 
surf the Web for general health-related 
material, but want sites offering specific 
information, and show little interest in using 
the Web to obtain general health-related 
information or products. Those more actively 
involved in their diagnoses and treatment 
decisions are more likely to use the Internet as 
a resource for information. 


Methods 


Data for the project were obtained from an 
online questionnaire hosted on the 
SurgeryDoor Web site[1] for the month of 
November 2000. In total 1,068 users 
answered the questionnaire, which 
represented 5 per cent of the 21,118 visitors 
(as denoted by unique IP addresses) to the 
site in November 2000. All questions were 
closed questions and covered a variety of 
topics including age, gender, income and 
family circumstance, views on site attributes 
and the health information needs of the user. 
A total of 32 per cent of respondents were 
aged under 34, 50 per cent were aged 
between 35 and 54 and 18 per cent were aged 
55 and over. SPSS was used to analyse the 
data. 


Platform background 


The consumer health information company, 
InTouch with Health, owns the publicly 
accessible Web site www.surgerydoor.co.uk 
(Figure 1). The site is frames-based, and has 
menus on both the left and right frames (with 
“content” displayed on a middle frame). 
When it was launched on 27 January 2000, 
SurgeryDoor became the UR’s first Internet 
health portal offering electronic versions of 
official NHS information, and the country’s 
biggest online health multi-store UK-specific 
health Web site (M2 Presswire, 2000). 
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The site features targeted sections 
providing health advice for groups such as 
’teens, carers, women, travellers and disabled 
groups. Site content is divided into the 
following sections, each of which appears as 
main content entries on a side-bar and 
contain a series of subheadings: 

+ Health Daily. Consisting of such things as 
a news and health alert, weather, tips 
(such as “what to look for when 
considering joining a gym”), and 
feedback and suggestions. 

* Medical. The first entry of which is 
“Emergencies”, and gives basic 
information on such things as bleeding, 
scalding etc. A “Symptoms Index” leads 
to the National Health Service online 
booklet Home Healthcare Guide, 
embedded in the SurgeryDoor window. 
Other content under the main “Medical” 
entry includes accounts of “Diseases in 
Depth”, a “Medical Dictionary” and a 
Prescription Drug Guide. 

¢ Healthy Living. Including such topics as 
Preventing Accidents, Dental Health 
Drugs and advice on alcohol 
consumption. 

« NHS & Benefits. Much of the content for 
this section is taken from the Government 
Benefits Agency and, indeed, appears to 
be pages directly from its own Web site. 

* Complementary Medicine. 

* Travel Health, Including vaccination 
advice and Traveller’s Health Kit (a list of 
items that may be useful during a 
holiday). 

* Community & Fun. Consisting of the 
interactive elements of the site and 
current health information. Content 
includes: “Patient experiences”, 
“SurgeryDoor Health Magazine”, 
feedback and suggestions, health surveys, 
quizzes and competitions. 

* Shopping. 

*  Today’s Selection of Features and Topics. 
Featured topics and articles such as 
“Living with Asthma” and “Hypnosis — 
its place in medicine today”. 


Results 


Satisfaction with the key attributes of the 
SurgeryDoor site 

Figure 2 shows how the SurgeryDoor users 
who responded rated nine attributes of the 
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Figure 1 SurgeryDoor Web site 
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Figure 2 Rating SurgeryDoor attributes - 
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site, including breadth of content, trust in the 
site, ease of navigation and advertisements on 
the site. They rated each characteristic out of 
4, where 4 was very satisfied and 1 very 
unsatisfied. Trust (3.4) was rated most highly. 
We might have expected this, as clearly these 
were users of the site and they would hardly 
be users if they did not trust the site, 
especially in the case of a health site. Breadth 
of content and site navigation were similarly 
highly thought of (3.3). The least satisfactory 
characteristics proved to be e-commerce 
opportunities (2.8), advertisements (2.7) and 
topicality (2.4). We might expect - - 


advertisements and shopping to score poorly 
— clearly some users were annoyed by these 
functions on a health site, and the 
dissatisfaction with currency shows the 
demanding nature of the digital health 
consumer. 


Categorising users according to site 
characteristics most rated 

Not all users will respond or relate to site 
attributes in the same way, and it would be 
valuable to type them according to the 
characteristics to which they do relate. Of 
course, some attributes might be seen to 
conflict with one another — for example, ease 
of navigational structure and breadth of 
content. There are conflicts between other 
characteristics as well; for example, our 
previous research (Nicholas et al., 2003) has 
found that where advertising is obtrusive 
there is a reduction in perceived 
trustworthiness in the site. 

A factor analysis, which identifies un- 
correlated or independent combinations of 
variables on the responses on how users rated 
each attribute, was conducted to see if 
meaningful groupings could be found. The 
results are reported in Table I. Three groups 
could be identified: those that rated content 
attributes, those that rated what we can term 
site facilities (e-mail, etc.) and those that 
thought most highly of system attributes, like 
speed of response/download. These three 
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Table | User types by attributes 


+ Content 

: Characteristic (21% of variance) 
Breadth of content 0.785 
Depth of content 0.757 
Trust in content 0.486 


Speed of delivery 
Navigation ease 
Advertisement 
Shopping facility 
E-mail facility 


w 


Factors, given in parentheses, scores greater than 0.4 includ 


groupings explained about 60 per cent of the 
variance related to these variables. 

Site visitors who favoured “content” were 
interested in breadth, depth and trust of 
content. Users most interested in “facilities” 
were happy with advertisements, shopping 
and e-mail facilities, while “system people” 
related favourably to the characteristics of 
speed of delivery and navigational ease. 

3 An important relationship was found 

= between the user’s score on the content 
attribute (breadth, depth and trust) and the 
number of different health sites visited. The 
user’s rating fell, as the number of sites visited 
increased (Figure 3). This suggests that there 
is a group of people that believe they have to 
visit many a site to find what they want. We 
call these people site checkers or evaluators. 
They largely do this checking on the basis of 
long experience with searching the Web, 


Figure 3 User scores on content attributes (breadth, depth and trust) over 
how many sites visited for health information 
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Attribute factors 


Facilities System 
(20% of variance) (19% of variance) 
0.773 
0.661 
0.627 
0.806 
0.451 


Notes: Principal axis factoring, Varimax rotation, Topicality dropped on the basis of a low communality score. 


led 


practice in making constant comparisons and 
a process of trial and error. The Web provides 
huge opportunity to “suck it and see” and 
these people take advantage of this. 

A relationship was also found between 
facility preference (i.e. for advertisements, 
shopping and e-mail) scores and age and 
gender. Figure 4 demonstrates the relationship 
with age: the user’s facility score decreased as 
age increases. This might result from a number 
of factors including the fact that older people 
might be less familiar with Internet technology 
like chat rooms and e-mails, have more fears 
about the security of online shopping and 
disapprove more of advertisements. In addition 
there was some gender bias with women being 
more likely than men to be happy with 
“facilities” (Figure 5). This relation was also 
repeated for system attributes. Women were 
also found to be happier than men with the 


Figure 4 User scores on facilities (advertisements, shopping. and e-mail 


and age) 





0.2 


0.1 


0.0 


-0.1 


Facility 


0.2 


0.3 


<15,000 18-22 23-23 30-44 45-59 


Note: Df = 5 375 f= 3.08 p = 0.01 





60+ 





Characterising and profiling health Web user and site types 


Aslib Proceedings 





Paul Huntington, David Nicholas and Peter Williams 


Figure 5 User scores on facility and system attributes and gender 
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system attributes — that is they tended to be 
more satisfied with speed of delivery and 
navigation ease. 

A relationship was also found between 
system attributes and the number of visits 
made to SurgeryDoor (Figure 6). Those 
people who visited the site less frequently (just 
once a month) scored lower on system 
attributes (speed and navigation). Logic 
suggests that these people were irregular users 
because of these factors (which they did not 
like) or perhaps they were just unhappy with a 
site with which they were not familiar. 
Furthermore, some users may have a slow 
Internet connection and consequently blamed 
the site rather than their own computer set-up 
for this. Attitudes towards system site 


Figure 6 System attributes (Factor 3) and number of times visited 
SurgeryDoor 
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attributes were also found to be related to the 
user’s health interest in the site (Figure 7). 
Carers and those users currently suffering from 
an illness were found to be much more positive 
with regard to the site’s system attributes. 


Health topics most sought 

Clearly content must be one of the most 
important reasons for using a health Web site. 
But precisely in what type of content are 
people interested? Figure 8 provides the 
details and shows user preference scores for 
12 health topics, such as New Treatments, + 
Natural Health, Pregnancy etc. Users rated 

each as very important, important, not so 

important or not important. Scores again 

were out of 4, where a 4 indicated that the 

topic was very important. The two most 

importantly rated topics were General Health 

information (3.3) and Diet (3.2). Poorly 

performing topics included information on 

Medical Conditions (1.8), Pregnancy (2.2) 

and Support Groups (2.3). This may well 

reflect user interest in coming to this 

particular site and maybe they get the latter 

information elsewhere. a 


Categorising users according to health 
topics sought 

A factor analysis was conducted to identify 
specific user groups by topic of interest. The 
results are reported in Table II. Four types of 
“topic” user groups could be identified: 


“alternative remedy users”, “staying fit and 


2 cc 


healthy users”, “keeping up to date users”, 


Figure 7 System attributes (Factor 3) and the user's health interest in the 
site 

















> 
0.3 ™ 
0.2 
0.1 
0.0 | 
-0.1 | 
-0.2 
| j 
03 : Long-standing iliness Carer Fit & healthy 
j Every day 2-3aweek Onceaweek 2-3amonth Once a month General interest Currently suffering Other 


Note: Df = 4, 842 f = 7.48 p = 0.000 








Note: Df = 5, 873 f= 2.73 p = 0.019 





282 


Characterising and profiling health Web user and site types 


Aslib Proceedings 





Paul Huntington, David Nicholas and Peter Williams 


Figure 8 Interest in finding out information about the following health 


topics 





Healthy Living 
Medical Condition 
General 
Prescription Drugs 
New Treatments 


Natural Health 


Complementary 
> Pregnancy 
Bg 

Support Groups 

Medical News 


Medical Research 


wre 























and “ill but want to know more users”. The 
combined factors accounted for about 60 per 
cent of the variance. 

Alternative remedy users rated the two 
topics Natural Health and Complementary 
Medicine most highly, while the fit and 
healthy group rated Healthy Living, General 
Health and Diet topics most highly. The third 
user type identified rated Medical News and 
Medical Research highly, which suggests that 
these users “want to keep up to date”. The 
fourth user type rated Prescription Drugs and 
New Treatments highly, which suggests a 
type of user who may be “ill but wants to 
know more” about what they have been 
prescribed and about new treatments. 

Age, gender and for whom the person was 
searching were significant characteristics of 


Table ll Types of user by topic interest 
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the alternative remedy group. Thus, 
alternative remedy users tended to be women 
under the age of 34 (Figure 9) and tended to 
search on behalf of friends and children 
(Figure 10). Previous research by the team 
has shown that information passed on by a 
friend is a poorly trusted information source 
and hence was unlikely to be used (Nicholas 
et al., 2002). However, these users were also 
using this information for themselves and for 
their dependants (Figure 10). 

The fit and healthy information user group 
was found to relate to the user’s current 
health status (Figure 11), and those who were 
currently healthy scored highly. These users, 
it seems, were accessing content so as to 
“stay” fit and healthy and to check health 
requirements etc. A low positive correlation 
(0.27) was also found between “content” 
attributes and this type of user, suggesting 
that these users were also interested in depth, 
quality and trust of content. 

The “ill but wants to know more” type of 
user was found, perhaps unsurprisingly, to 
relate to the person’s current health status 
(Figure 12), with carers, those currently 
suffering and those who had a long-term 
illness being more likely to feature in this 
group. This was also true to a certain extent of 
users identified as wanting to keep up to date. 
Those people with a long-term illness also 
sought to keep up to date with health 
information. Furthermore, those users 
searching for other people and for children 
also featured strongly in the “Il but wants to. 
know more” user group (Figure 13). 


Factor 1 Factor 2 Factor 3 Factor 4 
Alternative Staying fit and Keep up to IIl but want to know 
remedy healthy date more 
(17% variance) (17% variance) (16% variance) (10% variance) 
Healthy living 0.720 
General health 0.629 
Prescription drugs 0.721 
New treatments 0.559 
Natural health 0.845 
Diet 0.682 
Complementary medicine 0.814 
Medical news 0.734 
Medical research 0,861 


Notes: Principal axis factoring, Varimax rotation, KMO = 0.70, pregnancy, medical conditions and support groups 
dropped on the basis of a low communiality score. Factors, given in parentheses, scores greater than 0.4 included 





283 


Characterising and profiling health Web user and site types 
-Paul Huntington, David Nicholas and Peter Williams 





Figure 9 Alternative remedy user (Factor 1) and their age gender 
characteristics 
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Figure 10 Alternative remedy user (Factor 1) and for whom the user was 
searching 
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Figure 11 Staying fit and healthy (Factor 2) and current health status 
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Figure 12 II] but wants to know more (Factor 4) and current health 
status (also true for Factor 3) 
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Profiling users by the health site 
searched 

It is evident that different user types might be 
attracted to different types of health sites. A 
one-size, all-purpose health site that fits all 
user types and interests of users is unlikely to 
exist. People may be attracted to a site for 
style differences as well as site attribute 
factors and content differences. Also, old and 
young users may distinguish themselves by 
adopting different site-visiting behaviour. At 
present we do not have access to comparable 
data for other sites. However, we can, in the 
context of the current study, look at user 
differences revealed between SurgeryDoor 
users who have visited different combinations 
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of health sites. Here we will look at site 
attribute, topic interest and user characteristic 
differences between SurgeryDoor users who 
also use NHS Direct Online and with 
SurgeryDoor users who also visited Net 
Doctor. The aim of the analysis is to see if 
these differences reveal any information about 
the profiles of users for the various consumer 
health sites. 

A total of 29 per cent of respondents said 
that they visited just one site for their health 
information, 71 per cent of respondents 
visited two or more sites and 39 per cent 
visited three or more different sites. Figures 
14 and 15 show which site people visited first 
and which ones they additionally used. Not 
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Figure 13 Ill but wants to know more (Factor 4) and for whom the user 
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surprisingly, perhaps, about nine out of ten 

users said that SurgeryDoor was their first 

choice. NHS Direct Online, Net Doctor 

(www.netdoctor.co.uk), Medic Direct and 

Health in Focus were also mentioned as a first 

choice but the percentages were insignificant, 

between 1 to 4 per cent. These four sites, 
however, were significant as second sites 
visited. Net Doctor and NHS Direct Online 
were clearly important, each attracting 
around a third of responses as a second site 
preference. 

It was decided to restrict the between-site 
analysis to only those respondents who said 
that they had visited two or more sites. This 
was done so as to exclude from the analysis 
comparisons between those users just visiting 
one site and those visiting more than one site. 
In addition it was decided to exclude those 
respondents who said that they visited both 
NHS Direct Online and Net Doctor. 

A logistic regression model[2] was used to 
determine which site attributes, topics and 
user characteristics would explain differences 
between people using different combinations 
of sites. All the variables reveal something 
about users and variables indicated by the 
analysis will give an idea of the likely profile/ 
characteristics of the different user groups. 
Three models were considered: 

(1) a comparison between those 
SurgeryDoor users that used the NHS 
Direct Online site and those that did not; 

(2) a comparison between those 
SurgeryDoor users that used and did not 
use the Net Doctor site; and 
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Figure 14 First site visited 
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Figure 15 Second site visited 
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(3) a comparison between NHS Direct 
Online and Net Doctor users. 


Not all users here were SurgeryDoor users; 
we are simply grouping and comparing users 
of the two most important second sites visited 
with those that did not use these sites. Table 
IL lists the significant coefficients. 

There is an increased tendency for Net 
Doctor users on the SurgeryDoor site to be 
younger compared with other SurgeryDoor 
users. Net Doctor users were just under half 
as likely to be aged 55 and over and were 
about one-third less likely to be aged between 
34 and 54 compared with other SurgeryDoor 
users. The over-55s made up 18 per cent of 
total questionnaire responses, while the age 
group 35 to 54 made up about 50 per cent of 
responses. A total of 40 per cent of Net 
Doctor users were aged 34 and under 
compared with 29 per cent of other 
SurgeryDoor users. Age of user was also a 
significant difference between NHS Direct 
and Net Doctor users. The Net Doctor site 
seemed to attract a younger user type. Why 
should this be? This age profile might say 
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Table Ili Variables identified as explaining differences between different combinations of site use 


Compared with other SurgeryDoor users Net Doctor compared with NHS 


Direct + 
Net Doctor(1) NHS Direct(1) NHS Direct(1) 
Log odds Log odds Log odds 
N ratio SE ratio SE N ratio SE 
Family set-up 
Live alone 70 
Does not live alone 488 1.80t 0.33 
Salary (£) 
Less than 15,000 110 66 
15-22,000 137 1.03 0.30 80 0.93 0.35 
23-29,000 115 1.31 0.31 71 1.05 0.36 > 
30-44,000 119 1.76t* 0.30 85 1.08 0.36 
45-59,000 45 2.21* 0.39 27 2.38t 0.50 
- 60,000+ 32 0.75 0.49 21 0.41 0.56 
Age 
34 and under 197 127 
35-54 312 0.67* 0.19 184 1.66* 0.25 
55+ 79 0.64 0.29 39 1.52 0.39 
“Staying fit and healthy” type 
user 0.72*** 0.11 1.51** 0.12 175%" 0.15 
Navigation 132 : 68 
Important 284 1.35 0.23 173 0.74 0.31 
Very important 172 1.91* 0.26 109 0.51t 0.34 > 


Notes: Levels of significance (Wald’s Statistic): t < 0.1, *p < 0.05, ** p < 0.01; *** p < 0 .001 


something about how the site is presented. 
For example, there is a greater use of menu 
icons on this site compared with either 
SurgeryDoor or NHS Direct Online. Content 
is also considered an important attribute in 
attracting a younger user profile. 

In comparing NHS Direct Online users 
with other users, salary was found to be an 
important distinguishing factor. There was an 
increased incidence of a person being an NHS 
Direct Online user as income levels increased. 
This relationship is shown in Figure 16. It 
shows that 41 per cent of those SurgeryDoor 
users earning between £45,000-59,000 were 
also NHS Direct Online users but this was 
only true of 24 per cent of those earning less 
than £15,000. NHS Direct Online seems to 
attract users with a wealth profile. The wealth 
variable here may be an indicator for other 
variables that relate to it, for example, the 
user’s education and class. The NHS Direct 
Online site might be appealing to this type of 
user. Hence the NHS Direct Online may well 
be perceived as a more upmarket or a 
traditional, safe and conservative site — a 
broadsheet rather than a tabloid type of health 
site. NHS Direct Online users were also 
significantly more likely to rate general health 


and to a lesser extent medical news 
information as important or very important. 
Family characteristics was also important, 
with NHS Direct Online users being less 
likely to be “live alone” users. This was truer 
in a comparison with other SurgeryDoor users 
than with Net Doctor users. Family 
characteristics may be important in that 
families may be more likely to have regular 
contact with the NHS generally and might ae 


os 
Figure 16 Salary of respondents (grouped) by use of NHS Direct Online site 
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well search specifically for NHS Direct 
Online because of this. 


Information interest types ; 
Users identified as wanting to stay fit and 
healthy do differentiate or discriminate 
between the sites. Thus the staying fit and 
healthy user type tended to be more likely to 
be also an NHS Direct Online user and less 
likely to be a Net Doctor user. Figure 17 
describes this relationship and includes those 
users who had visited both sites. 

Those who used SurgeryDoor, who also 
used NHS Direct Online, were more likely to 


be a “staying fit and healthy” type user, while 


Net Doctor users scored poorly and this type 
of user was not attracted to the Net Doctor 
site. Perhaps Net Doctor users are not so 
interested in staying fit and healthy but are 
more interested in the health limitations and 
implications for their life style. Again this may 
say something about the content difference 
between the two sites (Net Doctor and NHS 
Direct Online) and how each site markets 
itself. 


System attributes 

The user rating of navigational structure was 
found to be a distinguishing factor. Thus, Net 
Doctor users were just over twice as likely to 
rate the navigational attribute as being 
important compared with either other 
SurgeryDoor users or users who additionally 
used NHS Direct Online. NHS Direct Online 
users were half as likely to rate navigation as 
very important. This may reflect real 
navigational differences between the three 


Figure 17 Staying fit and healthy (Factor 2) by site combination visited 
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sites. This argues that the market reaction is 
that the NHS Direct Online navigational 
structure is poor, while for Net Doctor it has 
been rated as good. Alternatively, this may say 
something about the attitude differences 
between NHS Direct Online users and Net 
Doctor users. NHS Direct Online users do 
not seem to rate navigational structure as 
important and this might imply a stronger 
brand attachment to NHS Direct Online. 
These users are not so interested in site 
attributes as in NHS Direct Online itself. 
However, for Net Doctor users navigational 
structure is important — these users seem to be 
site attribute-driven and may well represent 
more of a floating, less brand-loyal audience, 
and may well jump site to another offering a 
better attribute combination. 


Conclusion 


This paper has identified discrete sub-groups 
of health consumers and developed an 
understanding of the relationship between 
user types, user characteristics and site 
preferences. This type of research should be 
useful in informing designers of Web sites, 
especially with regard to menu construction. 
Patently, sites should now move on from 
broadcasting information generally to the 
“average” user to assisting specific groups of 
users to find what they are seeking. Thus, for 
instance, alternative medicine information 
seekers should be able to use the menu 
structure to quickly retrieve the information 
they need. Menus should reflect actual user 
interest rather than — as is too often the case — 
the producers’ or the site designers’ interest. 
To assist in this our study has provided user 
profiles for a range of consumer health sites 
and developed arguments around the sites’ 
information provision and presentation. Site 
producers and information providers can 
determine interests both by surveying users 
(and potential users, offline, if possible) and 
by analysing pages accessed — log analysis. 
Users were grouped according to the site 
characteristics favoured — content provided, 
facilities offered and system functions. Users 
were also identified by their topic of interest: 
“alternative remedy” user; “staying fit and 
healthy” user; “keep up to date” user; and 
“Tm ill but want to know more” user. 
Previous Internet health researchers have not 
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used these groupings. The key findings about 

these groups were: 

* With regard to user groups by site 
characteristics favoured, a relationship 
was found between the user’s score 
regarding the importance of content 
(breadth, depth and trust) and the 
number of health sites visited. It was 
found that the user’s content rating 
decreased as the number of sites visited 
increased, characteristic, one would have 
thought, of the “promiscuous” digital 
information consumer. Furthermore, the 
user’s facilities (advertisements, shopping 
and e-mail) rating decreased as age 
increased. There was some gender bias 
with women being more likely than men 
to be happy with advertisements, 
shopping and e-mail services. 

* Four types of Internet users were 
identified by their topic of interest: 
“alternative remedy” user; “staying fit 
and healthy” user; “keeping up to date” 
user; and “I’m ill but want to know more” 
user. Alternative remedy users tended to 
be women under the age of 34 and further 
tended to search for a friend or for a 
child. The “ill but wants to know more” 
type of user was found to relate to the 
user’s current health status and whether 
they were carers or not. Not surprisingly 
perhaps, those currently suffering from an 
illness and those who had a long-term 
illness were more likely to feature in this 
group, as well as those who were looking 
for information on someone else’s behalf 
and were interested in both new drugs 
and treatments. 

* It was possible to characterise users 
according to the pattern of health sites 
visited. Net Doctor users on the 
SurgeryDoor site were more likely to be 
younger compared with either other 
SurgeryDoor users or NHS Direct Online 
line users. The staying fit and healthy 
type of SurgeryDoor user tended to be 
more likely to be also an NHS Direct 
Online user and less likely to be a Net 
Doctor user. This may say something 
about the content difference between the 
two sites (Net Doctor and NHS Direct 
Online) and how the site attracts users. In 
was found that there was an increased 
incidence of a user being an NHS Direct 
Online user as income level increased. 
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Notes 


1 www.surgerydoor.co.uk 3W Marketing Ltd were 
responsible for gathering the information but not its 
analysis. 

2 Logistic regression was used in the second part of 
the analysis. Logistic regression is similar in 
“concept” to least squares linear regression, though 
its procedures, assumptions and underlying 
statistical model are different. Logistic regression is 
used whenever an outcome event can be classified 
into two populations. It is used here with regard to 
the outcome if a user made use of a secondary site 
(NHS Direct or Net Doctor): the user either did or did 
not. In particular the model is used to decide which 
users’ characteristics are predictive or significant for 
the outcome occurring. Logistic regression in 
addition estimates odds, that is it says something 
about the impact of each information source on the 
outcome. The odds are the likelihood of the 
outcome occurring, given the user-graded 
importance of each information source. 
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Introduction 


This research took place between January and 
October 2002. It was commissioned and 
funded by the MS Trust. Research into the 
information needs and information provision 
for people with Multiple Sclerosis (MS) was 
driven by a belief, supported by previous 
research, that information provision was 
inadequate (Robinson et al., 1996; Baker, 
1997) — also that information and knowledge 
are beneficial to the wellbeing, in terms of 
feelings of control, depression and reduced 
dependence on the health service, of people 
with MS (Somerset et al., 2003). 

The aims of this study were to: 

* provide systematically a comprehensive 
understanding of the information needs 
of people with MS; 

* determine what information needs were 
most important to people with MS; 

¢ determine what information was 
currently most difficult to find; 

* specify what information should be made 
available to people with MS; and 

* recommend how information should be 
made available to people with MS. 


The results of the research were expected to 
help determine the information policy of the 
MS Trust and also provide guidance, in terms 
of information provision, for service providers 
in the health and social services as well as 
other voluntary organisations and public 
information providers. This paper focuses on 
the information needs that were identified, 
explaining the subject-matter and why this 
information is important, the form in which it 
is required and the way it should be provided. 


Overview of multiple sclerosis 


There are approximately 85,000 people with 
MS in the UK. MS affects more women than 
men (approximately two women for every one 
man) and is generally diagnosed between the 
ages of 20 and 40. In parts of Scotland it is 
estimated that one in 500 will have MS; for 
the rest of the UK it is approximately one in 
every 1,000. 

In MS the myelin sheath around the nerve 
cells is attacked, the nerves become 
demyelinated and may not function as they 
should. MS can affect any part of the central 
nervous system and can potentially alter 
movement and muscle activity at several 
different sites, giving rise to a variety of 
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physical and psychological symptoms. 
Although it was first identified in the 1860s, 
the cause(s) of MS remain in dispute and a 
cure is elusive. 

MS is a particularly challenging disease in 
that it is highly unpredictable, and can result 
in a wide range of psychological and physical 
challenges that can vary from year to year and 
day to day — which can lead to a high degree 
of uncertainty and a need for information 
(Wilson et al., 2002). 


Previous research 


Little specific research has previously been 
conducted on the information needs of people 
with MS. Earlier research tends to be general 
and does not attempt to identify 
comprehensively and categorise the 
information needs of people with MS. It has 
been recognised, however, that people with 
MS need to make long-term, information- 
based decisions about their condition (Wollin 
et al., 2000). 

In 1996, in the UK, Robinson ez al. (1996) 
conducted a qualitative study of the needs of 
people with MS. Part of this study concerned 
information. They found that appropriate 
information was hard to find and use and that 
it was not available when and where they 
wished to access it. The information needs 
that were identified included: 

* information on the condition itself; 

* symptoms and managing symptoms; 

* current research; 

* benefits and entitlements; 

* insurance and mortgage advice; 

* transport for the disabled; 

* suitable holiday venues; 

* respite care; 

* medical and complementary therapies; 

* social and support groups; 

* voluntary organisations; 

* employment and statutory rights advice; 
and 

* home adaptations and aids. 


The report highlighted that individuals might 
have different information needs at any 
particular time due to different 
circumstances. The report concluded that a 
centralised source of information would be 
useful. 

In 1977 Matson and Brooks in the USA 
studied people with MS to determine how 
they had adjusted to the condition since 
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diagnosis. They concluded that most 
adjustment takes place in the first ten years 
after diagnosis. Based on their findings they 
proposed a four-stage model of adjustment to 
MS, in which information plays a role. Newly 
diagnosed patients entered stage 1 (denial) 
immediately. Here they sought information 
through different medical channels in an 
attempt to dispute the diagnosis. In stage 2 
(resistance) it was shown that people 
acknowledged the possibility of having MS 
and anxiously sought information about the 
condition and a cure. Patients also turned to 
others with the same condition and 
participated in groups or classes to garner any 
information that might be helpful. Stage 3 
involved telling other people about MS. Stage 
4 began when a person had fully accepted the 
condition. 

Matson and Brooks (1977) stated that 
information given by the consultant was seen 
as inadequate, perhaps because patients 
desired life-encompassing direction, which 
might be beyond the knowledge of the 
individual consultant. Most people with MS 
wanted practical information that would help 
them live with their condition. To find this 
information they used a variety of sources, 
with varying degrees of success. 

Stewart and Sullivan (1982), in the USA, 
found that information-seeking activity starts 
in the pre-diagnosis period and continues 
until a diagnosis of MS is confirmed. They 
also found that, because symptoms could be 
fleeting and might vary from one exacerbation 
(attack) to another, people engaged in 
“physician shopping” until they found a 
doctor who could diagnose their condition. 
People also sought information from popular 
literature and medical texts, as well as from 
health professionals, family and friends. 
Matson and Brooks (1977) noted that, in the 
post-diagnostic period, people with MS 
wanted practical information to help them 
live with their condition and sought 
information from other people with MS. 

Baker (1997) in the USA undertook a study 
of people who had recently undergone an 
acute exacerbation of MS. This study 
involved a questionnaire survey of 260 
women. The results helped to identify 
categories of information need and 
contributed to the questionnaire design in the 
current study. Categories of information need 
included: 


291 


Information needs of people with multiple sclerosis 
Mark Hepworth, Janet Harrison and Nicole James 





* physiological (general information about 
MS, nutritional literature, fatigue, bowel 
and bladder disturbances etc.); 

* psychological (dealing with a hidden 
disability, emotional behaviour, spiritual 
material etc.); 

* social (education, support for family 
members, branches for women etc.); and 

* economic (insurance, employer relations 
etc.). 


Baker revealed gaps in people’s knowledge 
about physical symptoms which people with 
MS were experiencing, emotions they felt, 
and drugs that could be used. Various sources 
of, and barriers to, obtaining information 
were identified. The results also suggested 
that health professionals needed to be aware 
of the continuing need for relevant, current 
and specific information to help people with 
MS retain their independence and to enable 
them to make informed decisions. 

Baker (1994) also found that people tended 
to fall into the category of either “monitors” 
or “blunters”, that is, people “either want 
information to handle an uncontrollable event 
or turn away from it because information is 
perceived to increase their level of anxiety”. 

A New Zealand-based study by Gregory et 
al. (1996) discovered that people with MS 
found generic information regarding MS 
readily available, but that practical 
information at a local level was frequently 
difficult to obtain. 

In a review of health-care preferences of 
people with MS in the UK, Somerset et al. 
(2001) found that three-quarters of the 
people surveyed reported lacking advice 
about at least one MS-related issue and 17 
per cent lacked advice in five areas. The most 
frequently consulted health professional was 
the GP, followed by the hospital consultant. 
The professional that most people said they 
would have liked to see was an MS specialist 
nurse. The study concluded that people with 
MS displayed a wide variation in their unmet 
needs, and that information about MS 
management (both conventional and 
unconventional), relevant tailored advice and 
access to appropriately skilled professionals 
should be components of high quality care. 

The literature does, therefore, provide some 
insight into the information needs of people 
with MS, their information-seeking behaviour 
and information provision. The needs can be 
seen to be broad and people with MS might 
not be successful in satisfying them. There 
were insufficient detailed empirical data to 
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enable a comprehensive definition of the 
information needs of people with MS. 

The research, discussed here, aimed to 
build on the previous research and to provide 
a more comprehensive view of the 
information needs of the MS population in 
the UK both at the time of diagnosis and 
thereafter. 


Methodology 


Qualitative and quantitative methods were 
used. A total of 11 focus groups involving 103 
people with MS of differing severity took part. 
These were held in England, Wales and 
Scotland in MS therapy centres. People were 
recruited through the centres, via local MS 
Society support groups and newsletters of the 
MS Trust and the MS Society. The 
population was therefore self-selected. 
However, each focus group session recruited 
people from one of three groups: people who 
needed no walking aids (59 per cent), 
wheelchair users (20 per cent), and those who 
used a walking-stick or walker (21 per cent). 
This breakdown corresponds to national 
figures on mobility for the MS population 
(Compston, 1998). At the start of the sessions 
respondents were asked to identify significant 
situations during their time with MS. Three 
to four of the most common situations were 
discussed by each group. Dervin’s (1999) 
sense-making methodology was used to 
structure the discussion. The information 
needs associated with situations were 
identified as well as barriers to access. This 
qualitative approach helped to understand the 
experience of people with MS and ensured 
that the categories of information need were 
grounded in the reality of the people with MS. 
These data, along with the published 
literature and input from stakeholders and the 
MS Trust, were used to inform the 
development of a questionnaire. The 
questionnaire was distributed to 4,100 people 
with MS through the MS Trust, MS Society, 
therapy and respite centres, newspapers and 
the Internet. To aid completion by the MS 
community, the questionnaires used large 
print and tick boxes; 2,030 questionnaires 
(49.5 per cent) were returned. No follow-up 
was instigated, partly because a 49.5 per cent 
response rate was felt to be good and also 
because of the time scale of the project. Again 
this was not a random sample;. people chose 
to be respondents. Of the respondents 1,509 
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were female, 512 were male and nine did not 
disclose gender (a ratio of 3:1, which 
compares with the national norm of 2:1). 
Respondents were aged between 16 and 75. 
The earliest diagnosis was 1937 and the most 
recent 2001; 553 people with MS were 
diagnosed within the last five years. Data 
gleaned from the returned questionnaires 
helped to quantify the demand for and value 
of categories of information, the difficulty in 
finding information and what was required 
generally and specifically at diagnosis. The 
survey also determined how information 
provision has varied over time. 

In addition audio-diarying was used to 
capture data on the day-to-day experience of 
a person with MS. Audio-diarying was 
undertaken by only five people, all who had 
severe symptoms of MS. Each person was 
given a small, easy to manipulate micro- 
cassette recorder and tapes and was instructed 
to record events, situations in their daily life 
where they needed to know something or 
wanted to find out something or had a 
problem that they needed to resolve. Any time 
a situation like this occurred they were asked 
to record what they were trying to resolve, 
why and whether or not it was resolved and, if 
it was, how. This technique of data collection 
was done primarily to see what kind of data 
would be generated rather than as a primary 
survey tool. This method did prove effective 
in providing detailed data on the situations 
the respondents confronted and their 
thoughts at the time — and generated greater 
detail than that generated by the other two 
methods. The audio data made apparent, for 
example, the range of symptoms that a person 
with MS could experience over a short period 
of time and also the importance, in terms of 
general wellbeing and quality of life, of taking 
an active role in life. Owing to the small 
sample the audio data were given little weight 
in terms of the overall findings. 


Findings 


Qualitative data 

Focus group sessions were all tape-recorded 

and later transcribed and then coded. A total 

of 132 generally problematic topics were 

identified by focus group respondents. Topics 

were categorised under the following 

headings: 

- A lack of knowledge when dealing with 
new situations associated with having 


Volume 55 : Number 5/6 « 2003 « 290-303 


MS, particularly at the time of diagnosis, 
but also subsequent situations such as 
finding out about state entitlements. This 
topic included discussion of how 
information is and should be provided. 
For professionals who are involved with 
MS it is hard to remember what it is like 
for a person who knows nothing about the 
condition or how to cope with a condition 
like MS. 

* General symptom management including 
finding out about symptoms and possible 
actions they could take that could help. 

* The difficulty of communicating with 
family, friends and colleagues and the 
need for appropriate information to help. 

¢ The problems associated with disabled 
access and transport and the lack of 
supporting information. 

* The need to make contact with and talk 
to other people with MS for both 
emotional and informative support. 


Discussion of these topics helped to identify 
information needs and also provided an 
insight into how information should be made 
accessible. 


Quantitative data 

The questionnaire design was informed by the 

data gathered from the focus group sessions, 

the published literature and discussion with 

the stakeholders and the MS Trust. The 

questionnaire focused on: 

* information provision and information 
needs at diagnosis; 

¢ determining the importance of categories 
of information and the difficulty in 
obtaining them; and 

+ the best format (paper, face-to-face, 
Internet etc.) for access and delivery. 


The aggregated data for all respondents were 
analysed. In addition the data from people 
who had been diagnosed in the last five years 
(553 respondents) were analysed separately to 
see whether the picture had changed over the 
last five years. It was found that information 
needs remained consistent and that 
information provision was still inconsistent in 
terms of topics covered and at provision. 
However, the number of people who said they 
received information at diagnosis has 
increased dramatically over the last 20 years 
(in 1980 only 20 per cent of respondents 
received information, whereas in 2000/2001 
70 per cent received information). In the last 
five years more information was provided on 
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two areas: drugs and MS nurses. This reilects 
the introduction of disease-modifying drug 
therapies and the increase in the number of 
MS nurses. 

At diagnosis over 50 per cent of the 
respondents wanted information on managing 
symptoms, drug treatments, disease course, 
physical symptoms, exercise, diet and 
information for the family. A distinction was 
found between the information needs at 
diagnosis and long-term information needs. 
This is discussed later. When respondents 
were asked whether or not they wanted to 
know about the different categories of 
information, they generally welcomed this 
information. However, not all people with 
MS wanted the same information or wanted 
to be informed at all. This may be because the 
severity of the condition can vary dramatically 
and people can experience very different 
symptoms. It may also be related to the 
information-seeking behaviour of the 
individual and intervening variables such as 
self-efficacy or coping strategies (Wilson, 
1999) and Baker’s (1994) conception of 
“monitors” and “blunters”. These individual 
differences may influence how people react to 
situations, whether people seek information, 
what information they require and how it 
should be provided. 


Discussion of information needs and 
implications for information provision: 
the need for a holistic approach 


The whole community 

Information provision needs to be targeted at 
three communities. These are people with 
MS, the general public (including the family, 
friends and colleagues), and state and 
voluntary service providers. As far as the 
authors are aware, with regard to MS, little 
has been done to tailor the provision of 
information about MS specifically to the 
different “communities” that are involved 
with MS. Each of these communities requires 
similar information but they require it in a 
different form and depth. Emphasis will also 
need to be placed on different types of- 
information for different communities. 


The public 

The public need a basic understanding of 
MS. Even people who currently have no 
direct contact with a person with MS need to 
be informed about MS. Some of the problems 
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experienced by people are due to the 
ignorance exhibited by the general public. For 
example, in the workplace, a person with MS 
who experiences severe fatigue in several cases 
was considered lazy. A lack of knowledge 
about MS could also influence employment 
practice. In the domestic situation the lack of 
understanding shown by the family also led to 
difficulty. In extreme cases this led to partners 
leaving due to misconceptions about MS. 
Again, through ignorance, some family 
members were concerned with whether MS 
was contagious. 

Information needs to be made available to 
the general public about MS through the 
national and local media. This would need to 
relate to the information needs of the 
employer, colleague, friend or acquaintance. 
In addition, probably a more effective way in 
the long term would be to incorporate in 
secondary schooling, probably under the 
heading of “citizenship”, information about 
MS. Ideally other conditions, particularly 
those that lead to care in the home, would be 
covered, since there is similar ignorance about 
other conditions and the help available 
(Hepworth et al., 2000). 

Information also needs to be designed that 
is appropriate for the family. Some headway 
has been made in this direction, for example, 
a book entitled My Dad’s Got MS 
(Hetherington and Young, 2001). This 
approach needs to be more comprehensive, so 
that similar information (although with 
differing content) would be developed that 
was designed for other family members. 

Access to this community information 
would need to be provided that was relatively 
simple, for example, not laden with medical 
terminology, clearly presented, and in a 
factual and positive manner (Ward and 
Cavanagh, 1997). This would include an 
overview of what MS is, the possible effects, 
medical support, social services support and 
voluntary groups. Information that helped 
people deal with specific situations would 
have to be provided separately and when 
appropriate. Geographical personalization, so 
that people are informed about local support 
groups and services, should be practised. 

If the person with MS is helped by someone 
who acts as an informal carer, then the 
information needs of the carer are going to be 
significant, particularly if the condition of the 
person with MS is severe. The informal carers 
would require far more information than 
other members of the public. They would also 
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have a need for different types of information 
over time, as the condition of the person with 
MS changed. Ideally, the situation of the 
carer as well as the person with MS would be 
monitored and appropriate information and 
support provided as necessary. This may 
sound ambitious from the perspective of 
service providers. However, the cost of 
enabling a person with a critical medical 
condition to stay at home, through 
information- and need-related services, is far 
cheaper than maintaining that person in state 
care. According to Walker and Dewar (2001), 
informal carers save the UK Government 
£57.4 million per year. 


Service providers 

Similarly, service providers would need 
general information about MS, the help and 
information that are available. The latter 
would help them to direct people with MS to 
informational and practical support. For 
example, a recent study of the information 
needs of hospital social workers, who work 
with adults leaving hospital, confirmed that 
they required overviews of medical 
conditions, medication and possible 
side-effects as well as information about 
legislation, policy guidelines and other 
relevant agencies (Harrison er al., 2002). This 
would help them to assess the kind of support 
the person would need and what can be 
provided. 

Service providers, including health- and 
social-care professionals and support groups, 
would benefit from training in the 
information needs of people with MS and 
how these might be satisfied. This would help 
them to understand what questions people 
with MS may have and hence what 
information was needed, when this may be 
required and how it should be provided. 
Service providers need to be aware of the 
variability of the condition and how the 
nature of MS may change over time. This 
implies that needs should be monitored over 
time and that health and social records need 
to be maintained, so that people do not need 
to instigate help from scratch when it is 
needed in the future. A better knowledge of 
the condition could also place service 
providers in a position to recommend services 
of which the person with MS may not have 
been aware or which they found difficult to 
request. For example, it took people with MS 
who had bladder problems a long time to find 
out about catheters, partly because they were 


Volume 55 - Number 5/6 « 2003 - 290-303 


embarrassed to discuss such personal 
problems and because information about 
bladder disturbances and catheters had not 
been found or given to them. Once they did 
know about these areas, and were able to take 
advantage of this equipment, it was easier to 
travel from the home. This had a big impact 
on their lives. 

In the focus group sessions diagnosis was 
the most common topic discussed. Where 
unsatisfactory experiences were described (49 
per cent of people attending the focus groups) 
these were associated with “a lack of 
support”, “poor attitude” and “providing no 
information”. However, it should be noted 
that being diagnosed is in itself a negative 
experience and is likely to colour a person’s 
judgement of the situation. Furthermore, the 
majority of people who did attend focus 
groups had been diagnosed more than ten 
years before, since when the situation has 
improved, Nevertheless from the 
questionnaire it was seen that 29 per cent of 
those diagnosed in the last five years felt they 
received no information at diagnosis. This 
indicates that information provision by health 
care professionals is not always successful and 
could be improved. Understanding what 
information is likely to be needed and 
knowing how to provide that information are 
not easy, bearing in mind that there are 
individual differences in terms of what is 
relevant to a person and what they actually 
want and also how they want it. Health 
professionals involved in diagnosis would 
therefore probably benefit from training in the 
information needs of people with MS and also 
how to provide information at diagnosis in a 
supportive and sensitive fashion. 


The person with MS 

Someone with MS should be seen as a person, 
and not only as a person with MS, who is 
likely to want to take an active role in life. 
Information should be provided that helps 
achieve this goal. Enabling people to 
participate actively in life as a result of access 
to information, and becoming better 
informed and more knowledgeable, whether 
in the home, workplace or elsewhere, has a 
positive impact on their self-esteem and 
general wellbeing. This may seem self- 
evident. It is, however, easy for service and 
information providers to focus on the 
condition and information specific to that 
condition rather than see the person with MS 
in the broader light and understand the 
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impact access to information can have. As a 
result they may underestimate the importance 
of information provision and fail to provide 
information that helps the person with MS 
really engage and participate in life. The 
audio tapes, for example, provided a good 
illustration of where an individual had 
become informed and had found out about 
adapting his car and sources of finance to do 
so, which meant he could take his daughter to 
graduation. This meant a great deal to the 
person with MS, including increasing his 
mobility, independence and, importantly in 
this case, being able to fulfil his parental role. 
Failing to take into account that the person 
with MS is, like any other person, fulfilling 
numerous roles, such as parent, wife, wage 
earner etc. and is having to deal with 
situations associated with these leads to 
simplistic and inaccessible information 
provision. First, without an understanding of 
the significance of information it is hard to 
prioritise what information to offer. Second, 
organisations that do not take this on board 
tend to provide information that only relates 
to the function of the organisation rather than 
the wider needs of the person. Third, research 
has shown that information seeking tends to 
take place when people have to resolve 
specific situations and tend not to seek 
information on topics that do not immediately 
concern them (Osiobe, 1998). When 
browsing for information people recognise 
useful information. partly because it obviously 
relates to their current situation (Bates, 
1989). The success of frequently asked 
questions is largely to do with this. The user 
does not have to wade through an A-Z of 
functions that they may or may not 
understand; in other words, they recognise 
the task and then the solution. A booklet 
called Tips for Living wih MS (MS Trust, 
2002), about how to deal with common 
situations experienced by people with MS, 
was popular, for similar reasons, among 
people with MS. These tips were collected 
from people with MS by the Department of 
Information Science at Loughborough 
University and collated, synthesised and 
published by the MS Trust. Therefore one 
way of providing information for people with 
MS is to identify common situations with 
which people have to deal, such as driving a 
car or dealing with an employer, and provide 
the necessary information. Information that 
enables someone to drive a car who has a 
physical disability would include, for 
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example, information about mobility 
schemes, car adaptations, possibly the 
usefulness of a catheter, disability access and 
so on. This “package” of relevant 
information, related to solving a particular life 
problem, would be easily recognised by the 
person with MS, when needed, and the fact 
that the information was integrated would 
mean that the person would not have to locate 
and interrogate numerous sources. However, 
as well as providing information in a way that 
relates to specific situations, information also 
needs to be provided in a discreet way, so that 
people can find out about a specific topic 
quickly. 

The information that people with MS 
stated they wanted at diagnosis is shown in 
Table I. What is evident from Table I is that, 
although there was a significant demand for 
most of these categories of information, there 
was a significant proportion of people who did 
not want that information. This highlights the 
complexity of information provision and 
raises questions, such as whether offering the 
information in a particular way would change 
the percentage of people who wanted it. 

After diagnosis, drawing on both the 
qualitative and the quantitative findings, it 
was determined that the following categories 
of information would help to satisfy the 
information needs of people with MS. These 
categories fall under two headings: 

(1) Information about MS. 

(2) Information that helps the person with 
MS interact with the world around them. 
This distinction may seem misdirected in 
that it could reinforce the mistaken 
perception of seeing the person with MS 
primarily in terms of their condition 
rather than as a person. As mentioned 
earlier, it would be good design practice 
to provide information through life 
situations as well as through a “directory” 
of topics. 


Information about MS 

In general people with MS should be 

provided with information about: 

+ MS, a general explanation. This would 
include an explanation of what MS is and 
how the body is affected. Diagrams can 
be very effective at communicating this 
kind of information. 

* MS research trials. This was one of the 
areas about which respondents found it 
either difficult or very difficult to find out 
(45.8 per cent). Owing to there not being 
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Table I What information would you have liked or not liked at diagnosis? 


What information would you have liked or not liked? 


Managing symptoms 

Possible drug treatments 
Indication of disease course 
Possible physical symptoms 
Information on exercise 

MS specialist nurse 

Information for family 
Information on diet 

Benefits and entitlements 
Physiotherapy 

Contact for group of newly diagnosed 
Alternative therapies 
Organisation addresses 

Possible psychological symptoms 
Counselling 

Work-related information 
Driving 

Occupational therapy 

Other 





Liked Liked (%) Not liked Not liked (%) 
1,334 65.7 685 33.7 
1,232 60.7 787 38.8 
1,121 55.2 897 44.2 
1,118 55.1 901 44.4 
1,093 53.8 926 45.6 
1,089 53.6 929 45.8 
1,072 52.8 947 46.7 
1,017 50.1 1,002 49.4 
995 49.0 1,024 50.4 
986 48.6 1,031 50.8 
948 46.7 1,071 52.8 
939 46.3 1,080 53.2 
913 45.0 1,106 54.5 
885 43.6 1,133 55.8 
804 39.6 1,215 59.9 
714 35.2 1,305 64.3 
637 31.4 1,380 68.0 
570 28.1 1,449 71.4 
115 5.7 1,906 93.9 
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acure for MS, at present, people with MS 
are keen to find out about new clinical 
trials, for example, the findings of the 
cannabis trials. People with MS were also 
interested in getting involved in trials but 
did not know how. 

Drug treatment. For similar reasons people 
wanted to know about drug treatments 
and whether new drugs were becoming 
available to people with MS and the 
regulations and costs associated with 
them; 44 per cent found this difficult or 
very difficult to find out. 

Symptoms. MS can result in many 
symptoms, such as loss of balance and 
mobility in general, sight problems, pain, 
severe fatigue and stress. Symptoms 
about which people highlighted it as 
being difficult to find out were balance 
problems, emotional changes and 
memory problems. People with MS can 
experience new symptoms. Each time this 
occurs they need an explanation of what 
is happening, how serious it is and what 
can be done. Support groups proved 
particularly popular, because people 
could ask others about a symptom and 
someone else who had experienced it 
would inform them. Respondents did 
indicate that one could not always rely on 
the views of other people with MS. 
Another route for discussing symptoms 
(and other topics), that the MS Trust 
have offered, was via electronic chat 
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sessions where people can exchange 
information. An expert was also present 
who could give a professional view and 
act as a moderator. 

Treatment and management of 
symptoms were highlighted as a separate 
topic. There is a distinction between 
knowing what something is and how to 
treat or cope with it. This kind of 
information can be provided ‘iin a factual 
way. However, where coping is more 
relevant than treatment, through the use 
of drugs, physiotherapy etc., information 
provision would need to be broader and 
require thoughtful design in that it should 
be motivating and include coping 
strategies, 

Information on the full range of 
symptoms experienced by people with 
MS and their treatment or management 
is therefore required. 

Prognosis. Providing information on this 
area requires sensitivity. Some health 
professionals, in conversation, say that 
people do not want to know how the 
condition could develop either at 
diagnosis or at any other time. This was 
borne out to some extent from our 
findings (47 per cent did not want 
information). It is also an area of 
uncertainty due to the unpredictable 
nature of the disease. However, 53 per 
cent of the respondents who had been 
diagnosed within the last five years 
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wanted, at diagnosis, an indication of the 
disease’s course. It is likely that this 
response would depend, to some extent, 
on the condition of the person and also on 
how they are dealing with the situation 
emotionally. Prognosis can help people to 
plan and take control assuming a suitable 
state of mind. 

Physiotherapy. A total of 80 per cent of all 
respondents thought it important or very 
important to know about physiotherapy. 
It was thought generally less difficult to 
find out about than other topics. 
Complementary therapy. A total of 68 per 
cent of respondents thought it was 
important to know about complementary 
medicine. This includes hyperbolic 
oxygen as well as other complementary 
approaches such as massage, acupuncture 
and complementary medicines. One 
concern expressed by respondents and 
the MS Trust was that there is little 
quality control in this area and now, 
through the World Wide Web, people can 
become aware of and use “alternative” 
approaches that are harmful. 

Nutritional advice. There is evidence that 
diet can have an effect on MS symptoms 
and that certain foods exacerbate the 
negative symptoms of MS. As a result 
(and also because drug treatments may 
not be effective) there was a lot of interest 
in diet. 

Health and fitness. Similarly, maintaining 
health and fitness is thought to be 
beneficial. People with MS wanted 
information about what exercise could 
help and how to take part in appropriate 
exercise. Aquatics, for example, is felt to 
be particularly appropriate, because the 
water provides support for the weakened 
body and also reduces body heat, which 
has an adverse effect on people with MS 
(National Multiple Sclerosis Society, 
2003). 

Emotional changes. This was a topic about 
which people with MS found it difficult to 
find out. As well as suffering from 
depression and mood changes people 
with MS can also experience severe 
fatigue that can have a negative impact on 
people’s state of mind. Being aware of the 
likely emotional changes that can take 
place and also ways that can help to 
ameliorate them is therefore important. 
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Information that helps the person with 

MS interact with the world around them 
Information that helps the person with MS 
interact with the world around them is not 
directly concerned with the medical 
condition. This includes information about 
the following: 
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Service providers. Health service, social 
service, benefits agency, non-government 
and voluntary organisations, and the 
services they provide. Previous research 
with informal carers (Hepworth et al., 
2000) and recent work with people with 
Alzheimer’s disease (Pritchard, 2003) 
highlight the difficulties people have with 
dealing with the various bodies that can 
provide help for the person with MS. The 
lack of knowledge people have about state 
and non-government organisations 
means that when people have a need they 
find it very difficult to understand exactly 
what organisations do, the responsibility 
of different roles, such as a community 
nurse, and how to instigate services 
(including how to fill forms 
appropriately). Organisational charts 
need to be made available to help people 
contact the right people. In addition 
advice on how to complete application 
forms needs to be provided. Increasingly 
we are seeing attempts to provide more 
“joined-up” services. This will be helpful 
in that organisational boundaries will be 
less likely to get in the way of information 
provision. 

Support groups. These groups played a key 
role for many people with MS. It should 
be noted that some people were not keen 
on joining groups. This was partly 
because they did not want to see people in 
a worse condition, as this indicated what 
the future might bring, or they did not 
want to be constantly reminded of their 
condition. Other people found support 
groups extremely useful for exchanging 
information, learning about their 
condition, identifying additional support 
and providing emotional support. 
Generally the atmosphere in therapy 
centres, for example, was extremely 
supportive and helped people maintain a 
positive state of mind. 

Aids and appliances, This was a category of 
information that related to people who 
had more severe MS and had, for 
example, problems with mobility. Again 
finding out about aids and appliances and 
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their relative merits as well as potential 
sources of funding was not easy. In fact 
once one knows where to look there is a 
wealth of information on this area and 
Disability Living Centres, that show a 
wide range of aids and appliances, are 
found in most large towns. 

Facilities for the disabled. Similar to the 
above, this category is not specific to MS. 
However, severe MS does tend to be 
associated with problems that require 
access to facilities for the disabled. 
Finding out sufficient detail about 
disabled access is not easy and in the UK 
disabled access is poor. This situation is 
primarily due to ignorance about 
disability among the general public and 
presumably weak legislation enforcing 
good design. A conversation with people 
with MS about a building that was 
supposed to be “disabled-friendly” 
indicated that it was not. They thought 
that disabled people should be involved in 
the planning and design. Poorly designed 
buildings provide a severe barrier for 
people with disabling MS who want to 
take an active part in life. Hence 
information needs to be made more 
readily available on disabled access and 
facilities in public buildings. In addition, 
people with MS with problems with 
mobility also needed to find out about 
making their own home “accessible” and 
possible sources of finances to do this. 
MS and work and the issues associated with 
giving up work. Being able to work and 
bring in an income is, of course, of 
fundamental importance to most people. 
It was, therefore, not surprising that 
people with MS wanted information 
about work, their rights, employer 
obligations, dealing with their employer 
and also what would happen and what 
they could do, if they had to give up 
full-time employment. 

Communicating about MS with family, 
friends, colleagues and the public. As 
mentioned earlier, information for the 
family, friends, colleagues and the public 
is very important. It was difficult for 
people with MS to communicate with 


- others about their condition. People with 


MS need information that they can give 
to such people. Material specifically 
designed for this purpose would therefore 
be useful. 
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+ Leisure activities including, when necessary, 


provision for the disabled. Information 
about leisure was not the most important 
type of information that people wanted. A 
total of 31 per cent of people with MS did 
state that they found it difficult or very 
difficult to find out about appropriate 
leisure activities and holidays. It was 
important for people with MS to know 
what was possible, rather than assume 
that it would be impossible to take 
holidays and enjoy leisure activities, 
especially if disability was an issue. 


These categories of information are similar to 
those generally recognised as being required 
by people with an illness (Duman, 2003). The 
categories listed here cover a wider range of 
topics and include information not directly 
related to the condition but are important for 
the person’s wellbeing and ability to 
participate in life. 


How should information be made 
accessible? 


From the authors’ experience of conducting 
studies of people’s information needs and 
information-seeking behaviour it is felt that 
there is still considerable scope for 
understanding how information should be 
provided and what factors influence whether 
information is taken on board and has an 
impact on the consumer of the information. 
As can be seen from the data provided here 
within a community, such as people with MS, 
the demand for different types of information 
is varied, as is how people want information. 
Generally, with regard to people with MS in 
this study, a number of information 
characteristics were important for successful 
information provision. 

Information should be provided: 

* In a positive manner. People with MS 
stated that the majority of information 
they received emphasised the probable 

“negative outcomes rather than stating 
what can be done. Publications that 
talked about how people coped with their 
MS, and specific conditions, were very 
popular. Information can be provided 
about a particular condition. If the 
information is combined with positive 
instruction and “tips” and reassurance 
about the things that can still be achieved, 
then the information is more likely to be 
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absorbed and a more positive frame of 
mind maintained. This can help a person 
take a proactive and positive approach to 


managing their condition. . 


In a way that helps and motivates people to 
take action. This is similar to the last 
point; however, more thought is required 
into what kind of information is going to 
enable people to take action. On the one 
hand, as noted above, information can be 
phrased in a positive way and practical 
instruction given that will help people 
cope. It may also be the case that, at 
different points in time and depending on 
the frame of mind of the individual, 
information should be provided in 
different ways. In counselling, for 
example, the theory behind motivational 
interviewing techniques (Miller and 
Rollnick, 2002) states that people go 
through a cycle of stages when they are 
trying to change their behaviour. These 
stages are contemplation, determination, 
action, maintenance and relapse. During 
these stages they are open to certain types 
of information: factual, positive, 
supportive etc. They argue that helpful 
information will correspond to these 
stages or states of mind. It is possible that 
similar or identifiable states of mind are 
associated with dealing with a condition 
such as MS and a greater understanding 
of how changes in people’s state of mind 
affect their need for information would be 
beneficial. 

That corresponds to the physical, 
psychological and social situation of the 
person with MS. People tend to want 
information that relates to their specific 
situation. This needs to be taken into 
account when providing information. In 
other words, people need to be able to 
identify information that relates to their 
situation and receive information that is 
relevant or takes into account their 
situation. Technological approaches to 
doing this will be discussed later. 

In a way that is sensitive to the desire for 
information and its possible impact on 

the individual. As noted, not all people 
demand all categories of information or - 
any at all. This has to be taken into 
account by the information provider and e 
catered for accordingly rather than that 

_ one should assume, for.example, one 
message for all. As discussed earlier, this 
requires the information provider to be 
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sensitive to the needs of the individual, 
whether in face-to-face situations or when 
interaction is through a machine. 

Using the most effective means of 
communication. This will vary depending 
on the nature of the information and the 
cognitive and physical ability of the 
person with MS. For example, face-to- 
face communication was found to be 
more appropriate for sensitive or complex 
issues, such as balance problems, bowel 
and bladder disturbances, drug 
treatments, emotional problems etc. 
Leaflets and booklets were thought to be 
more appropriate for topics such as access 
and transport, aids and appliances, 
benefits and entitlements, working and 
MS etc. Newsletters were felt to be most 
appropriate for information about 
research in general and specific research 
trials. Discussion groups helped answer 
questions and enabled people to share 
experiences. Different types of 
information resources have different 
characteristics that may or may not be 
attractive to different users. For this 
reason access should be provided to 
different sources of information that 
satisfy different needs, for example, 
academic and professional articles, 
popular press, experts and other people 
with MS. Each of these sources has 
different characteristics in terms of the 
depth, detail and style of presentation 
that will be more or less appropriate 
depending on the information need of the 
person with MS and their preference for 
how information is presented. For 
example, other people with MS are 
valued for their insight into specific 
situations and their ability to provide 
relevant information in a way to which 
others can relate. Academic articles are 
popular with independent learners who 
want detailed information about research, 
that has gone through a peer review 
process, and have the ability to 
understand the terminology used and feel 
unthreatened by the academic style of 
presentation. People should therefore be 
given a choice as to what type of material 
they can access. 

Using different presentation techniques to 
cater for individual conditions that a person 
with MS may experience. Overall 60 per 
cent of respondents stated that MS 
affected their information seeking and 
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finding. For example, severe fatigue was 
cited by 47 per cent of respondents as 
affecting their information seeking. This 
was followed by mobility (45 per cent). 
Problems with concentration also affected 
access to information. The design of 
information provision needs to take this 
into account in both paper-based and 
electronic domains. People with high 
fatigue and low concentration levels are 
going to require information in a succinct 
form with emphasis on a graphical rather 
than textual presentation of information. 
Mates (1995) has shown that good 
design, for example, using large print and 
contrasting colours, makes information 
easier to interpret. The quality of design 
of information for people with MS is very 
varied. Different organisations provide 
different as well as the same information 
that is aimed at different audiences, such 
as health professionals or people with 
MS. Some follow good design standards, 
others do not. Greater cognisance of good 
practice by information providers is 
necessary. For example, notice needs to 
be taken of the work of the Centre for 
Health Information Quality (2003). The 
latter have developed criteria for checking 
the quality of public information. The 
Welcome to Plain English Campaign 
(2003), as well as the King’s Fund book 
(Duman, 2003) on producing health 
information, also provide guidance. In 
addition Web designers need to conform 
with good practice. In particular, the 
World Wide Web Consortium’s Web 
Accessibility Initiative should be taken 
into account (WAI, 2003). The NHS 
have also sponsored the Discern project 
(Discern, 2003), which provides a tool to 
help to evaluate Web sites. Currently, 
much information available via the World 
Wide Web is difficult to access by people 
with physical disabilities such as poor 


sight due to poor design of Web sites and - 


their inability to be read by screen readers 
(Scherer and Galvin, 1996; Oppenheim 
and Selby, 1999). In addition, small font 
and a lack of contrasting colours inhibit 
access. Hence information provision 
needs to be designed, so that people can 
access it despite their disability and can 
take advantage of adaptive/“assistive” 
technologies. Legislation, concerning 
electronic access to information, will 
come into force in 2004 (RNIB, 2002) 


Aslib Proceedings 
Volume 55 - Number 5/6 - 2003 - 290-303 





that should help to improve the current 
situation. 


Further research 


Further work is required into the effect of 
individual differences (psychological and 
physiological) on the desire to learn and to be 
informed and the use of information. 
Research is also needed into how technology 
can help to provide personalised information. 

A number of technological approaches have 
been taken to offer personalised Web pages. 
Bental et al. (1999) summarised a number of 
projects in this area. At the University of 
Brighton the ICONOCLAST project 
(ICONOCLAST, 2003) investigated how . 
appropriate style and layout of documents can 
be determined through interaction between 
the author and the system. Patient leaflets 
were analysed to understand effective layout. 
At Loughborough we are currently 
investigating how Extensible Mark-up 
Language (XML) can be used to provide 
personalised information delivery based on 
the user’s situation. 

XML is a sub-set of Standard Generalised 
Mark-up Language (SGML). XML is 
designed to describe data and what data are. 
It is not a replacement for Hypertext Mark-up 
Language (HTML). HTML is designed to 
display data and to focus on how data look 
using pre-defined tags. In XML the publisher 
defines the tags. For example, <prognosis> 
could be a tag that is then used to identify 
material about prognosis. A browser does not 
know how to display an XML document. 
Therefore there must be something in 
addition to the XML document that describes 
how the document should be displayed. This 
is the role of XSL. XSL consists of three 
parts: XSLT (a language for transforming 
XML documents); XPath (a language for 
defining parts of an XML document); and 
XSL Formatting Objects (a vocabulary for 
formatting XML documents) (W3C, 2003b). 

XSL is a language that can transform XML 
into XHTML, filter and sort XML data, 
define parts of an XML document, format 


-XML data based on the data value (like 


displaying negative numbers in red), can 
output XML data to different devices, like 
screen, paper or voice (W3C, 2003a). 

A screen was designed that allowed the user 
to make choices and indicate their situation, 
condition and information needs. For 
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example, they could request large font and 
specify that they suffered ftom severe fatigue 
and that they had recently been diagnosed. 
This generated XSL scripts that then worked 
on the XML document. For example, having 
selected large font, the following XSL code 
was generated:> 


< xsl : variblename = 2font” > 5 
< /xsl : varible > 


where the number five signifies large font. 
Other characteristics of documents such as 
subject-matter, level of depth, text density, 
relevance to stage of MS, would have to be 
defined and a taxonomy would need to be 
developed that describes documents and their 
content, so that appropriate content can be 
generated when required. 

If a portion of text in the XML document 
was tagged <start stage=“a” den=“ab”>, this 
meant that, when the XSLT code was 
generated that stated the person with MS was 
at stage “a”, i.e. recently diagnosed and 
wanted information in brief “ab”, then a 
positive motivational piece of brief text was 
displayed from the XML document. XML 
and XSLT therefore work in conjunction, 
where XSLT is the programming language 
that develops the logic using standard 
conditional statements that act on the XML 
document. Using this approach, one highly 
structured document can be presented in very 
different ways to the reader. 


Conclusion 


This paper has described the information 

needs of people with MS. Possible solutions 

were also discussed. The need for people with 

MS to have access to information about MS 

as well as to information that helps them 

interact with the world around them was 

highlighted. To improve the situation of 

people with MS, information and training 

should also be made available about MS, and 

the information needs of people with MS, 

to the general public and service providers. 

The information needs of people with MS 

was seen to be complex because: 

* the needs change over time; 

+ the range of topics they may or may not 
need to know about; 

* variations in the desire for information; 
and 

* there are differences in how information 
is needed due to the physical and 
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psychological state of the person with 
MS. 


To achieve the level of personalisation 
required to effectively provide information to 
these communities, evolving technology can 
be exploited as well as standard methods of 
information provision and exchange. Our 
current understanding of exactly how the 
state of the individual affects their need for 
information and what this means for the 
design and delivery of personalised solutions 
is still relatively crude and requires further 
investigation. For example, what is the most 
appropriate taxonomy to describe different 
types of information held in KML 
documents? What is the most appropriate way 
of communicating different types of 
information about MS to a person 
experiencing severe fatigue or lack of 
concentration? Can information be presented 
in such a way that a “blunter” becomes a 
“monitor”? How would it be best to package 
information for the public and family? How 
can information be packaged in such a way 
that people with MS can easily recognise the 
situations with which they are dealing with 
and then find all the appropriate information 
that will help? For many of these questions we 
can envisage possible answers. However, to 
achieve a better understanding, individual 
differences need to be studied in more detail; 
an appropriate methodology for storing 
information needs to be developed; and 
access to this information through a range of 
techniques needs to be enabled. Having done 
this, these solutions need to be evaluated in 
terms of their impact on the MS community 
and the other target communities. The 
lessons learned through this kind of exercise 
would enable the full-scale implementation of 
relevant and personalised information 
solutions to people with MS. This work 
would also be relevant to other information 
providers who want to provide similar 
information services on different topics. 
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relationship. Although a questionnaire survey was used 
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method was a success in terms of the amount and richness 
of the data accrued. Results suggested that the Internet is 
exploited in a wide variety of ways, by users acting in a 
number of roles — patient, intermediary or professional. 
Some health professionals are now so comfortable with the 
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understanding of issues raised, and showed a (healthy?) 
scepticism regarding the information provided. Many users 
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Introduction 


Health information has permeated the 
Internet. Electronic health information is 
becoming ubiquitous. Quite apart from the 
Government’s NHS Direct presence on the 
Internet there exist thousands of health Web 
sites of commercial, voluntary and individual 
origin. Medical support/discussion 
newsgroups have also flourished. For those 
without Web access touch-screen kiosks can 
be found in health centres, supermarkets and 
libraries, and the Internet itself is now 
accessible on touch-screen systems and digital 
television. Despite the huge economic and 
political investment in digital health 
information-provision there has been little 
evaluation of the investment in terms of its 
impact on the health consumer. The CIBER 
group at City University are working on a 
major Department of Health (DoH)-funded 
research project, “Digital Health” (Nicholas 
et al., n.d.), on the use and impact of the 
remote health information initiatives 
mentioned above. This paper reports on one 
study within this research programme: a 
qualitative examination of use of the Internet 
by the general public, using the somewhat 
novel method of an online survey that 
attempted to capture qualitative data from a 
small number of open questions. Much 
attention is afforded to the methodology, 
given its comparatively unusual nature. 


Aims and objectives 


The main aim of the research was to enrich 
data accrued from both computer transaction 
log analysis (see, for example, Nicholas et al., 
2002a) and questionnaire surveys employing 
principally closed questions (Nicholas et al., 
2001a, b, 2002a, b) measuring the use and 
impact of the Internet on the general public. 
The study was interested principally in the 
extent to which, and in what ways, the 
Internet was being employed with regard to 
people’s health. The objectives of the study 
were to examine: 
reasons behind people’s decisions both to 
look for health information and to consult 
the Internet for this purpose, and what 
drew people to the NHS Direct site in 
particular; 
* how they use the Internet with regard to 
health (i.e. the number and type of 
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Internet sites accessed, and for what kind 
of health information); 

e possible impacts on dealings with the 
doctor as a result of finding and using 
information acquired from the Internet; 

+ their perceptions of information in terms 
of quality, authority and trustworthiness. 


Methodology 


An open-questioned survey method was 
employed for this study, posted on to the 
NHS Direct Web site. It was hoped that the 
more qualitative data accruing from open 
questions would, as Silverman (1999, p. 8) 
states in his review of qualitative research, 
“provide a ‘deeper’ understanding of social 
phenomena than would be obtained by purely 
quantitative data”. Often a qualitative 
methodology is used when little is known of 
the phenomenon under scrutiny (Glasser and 
Strauss, 1967). In the current case, however, 
the territory is not entirely virgin, as we have a 
good idea of the “information landscape” by 
virtue of log statistics and survey-data 
(Nicholas et al., 2001a, 2002b). However, 
qualitative research addresses not only 
relatively narrow questions about the use of 
and opinions about the particular system or 
service, but with the full context in which an 
information need arises and is met. Moore 
(2000, p. 121, original emphasis) claims that 
“quantitative data are very good at telling you 
what is happening, qualitative provide an 
insight into the deeper question of why?” 
Previous research by the group, particularly 
work on the use and impact of the Internet 
(e.g. Nicholas, 1996; Nicholas et al., 1999), 
has confirmed assertions (Dervin 1992; 
Wilson, 1990} that often the perspectives of 
information professionals and researchers are 
not those of the user community. It is vital, 
therefore, that in researching the needs of 
those for whom the service — be it an 
information kiosk, Web site or digital TV 
initiative — is targeted must be free to describe 
their situations, needs and experiences, “in 
terms of their own frames of reference ... on 
their own terms” (May, 1997, p. 112). The 
present research employed a loose, semi- 
structured questionnaire format. This 
approach allowed people “to answer more on 
their own terms”, but still enabled the 
researcher to work with a topic framework. 
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An important issue, with regard to the 
methodology, is that of the quality of 
responses. Work has been done on this. 
Kiesler and Sproull (1986) compared 
response patterns with a paper questionnaire 
and an electronic questionnaire. They found 
that respondents who answered the e-survey 
made fewer mistakes and gave richer answers 
to open-ended questions. 

Research has also shown that online 
questioning results in fewer items being 
omitted by respondents (Kiesler and Sproull, 
1986; Schaefer and Dillman, 1998; Sproull, 
1986). The biggest difference between online 
and offline methods, however, is that 
electronic surveys produce richer responses to 
open-ended questions (Mehta and Sivadas, 
1995; Bachman et al., 1996; Schaefer and 
Dillman, 1998). Responses to open-ended 
questions in online surveys tend to be longer 
and more revealing than those generated in 
standard postal, self-completion surveys 
(Taylor, 2000). 

Finally, online research may also be 
effective in addressing sensitive issues. 
Respondents seem to be more willing to 
reveal information about their experiences 
with sensitive conditions (e.g. anxiety 
disorders, ovarian cancer, incontinence, 
erectile dysfunction) in online surveys than 
face-to-face. Moon (1998) found that 
consumers revealed a great deal of personal 
information when completing a computer- 
mediated survey. Illingworth (2001) asserts 
from her research on the Internet that the 
familiarity developed “is, at times, difficult to 
relinquish”. The questionnaire for the present 
study did not seek to obtain specifically 
intimate or sensitive information about 
respondents’ health information needs, 
although, of course, openness and honesty 
were desired, and a certain level of intimacy, if 
necessary, with regard to topics researched. 

The questionnaire was posted on to the 
NHS Direct site for three weeks in November 
2002. It began with an introductory and 
explanatory paragraph, which explained the 
research and indicated that respondents could 
interpret “health” in as wide or narrow a way 
as they wished. Seven “open” questions 
followed, which addressed the objectives 
outlined earlier (i.e. “Why do you turn to the 
Internet for health information?”). Subjects’ 
age range, gender and occupation were also 
requested. Questionnaire responses were 
received by e-mail, via a “mailto” link. In 
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many cases (24 of 42) subjects were asked 
follow-up questions to clarify or expand on 
initial answers. Only seven people, however, 
entered into this dialogue, presumably feeling 
that they had said all they wished to say on the 
subject. 

E-mail texts were “framework”-analysed 
(Richie and Spencer, 1994). This approach 
involves a systematic process of filtering and 
sorting material into themes, and has been 
used often in health/medical research (e.g. 
Buckland and Gorin, 2001; Leydon et al., 
2000). As the questionnaire had already pre- 
determined seven main themes that interested 
the researcher, the process was relatively easy, 
although, of course, some respondents 
strayed from the question theme or did not 
address it. 

Two researchers independently coded and 
sorted the data and compared their resulting 
“frameworks”. There was little disagreement. 
Indeed, the only discrepancy was with regard 
to the issue of displacement of sources. 
Reports of visiting doctors less often were 
described as a displacement of source by one 
coder and as an impact on dealings with the 
doctor by another. It was agreed that the 
coding depended on the definition of 
displacement, and that the researchers would 
honour respondents in that those who 
discussed this in response to the section on 
displacement would be cited in that section, 
whilst views of those who saw it in terms of 
dealings with the doctor would be discussed 
there, with appropriate cross-references. 


Sample 


Unlike postal or face-to-face questionnaires, 
respondents are not targeted with an online 
questionnaire posting, beyond the fact that 
they will be users of the particular site chosen. 
A total of 42 people responded to the 
questionnaire. This was not considered 
sufficient to generalise answers in terms of 
age, gender or occupation, although it did 
generate over 50 pages of qualitative word- 
processed data. For the record there were 25 
male and 17 female respondents, the modal 
age groups being 30-39 (female) and 40-49 
(male). Most respondents declined to reveal 
their profession. Those who did included four 
health professionals, an information 
professional, and two journalists. 
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Results 


Reasons for using the Internet 

A great number of reasons were given for 

turning to the Internet for health information. 

Respondents answered in terms of: 

+ the advantages of the Internet over other 
sources of information; 

* the role of the Internet in their 
information seeking; and 

* the topics they researched. 


The most popular advantage of the Internet 
was its perceived convenience: “It’s open 24/7 
and right in my own home. I don’t even need 
an appointment. No waiting!” Part of this 
convenience was the use of health information 
to circumvent the need to visit a GP: “people 
may feel less inclined to visit the doctor, if 
they have access to accurate health info [sic] 
from home”. Although this purpose was only 
specifically mentioned by three respondents, 
others referred to this in answer to a later 
question looking at the effect of information 
on respondents’ dealings with their doctors. 

“Convenience” seemed to be 
interchangeable with “anonymity” for some 
respondents, in the sense that anonymity is, 
itself, one of the conveniences: “It is 
extremely convenient to use my own PC in 
the privacy of my home.” and “A readily 
available (24/7 at home) and anonymous 
source of information.” Apart from the 
linkage of the home with privacy, those who 
mentioned anything concerned with 
anonymity or confidentiality did so in 
somewhat general terms (such as simply 
stating that one could look up information 
privately). One person confirmed that this 
was for information about which he felt 
“unable to ask doctors”. 

Surprisingly, only three respondents (in a 
sample that included health and information 
professionals, journalists and company 
directors) mentioned currency as a possible 
attraction of the Internet. 

Perhaps, also surprisingly, the depth and 
breadth of information on the Internet were 
only cited in the first instance by two people. 
It may be that, these days, this aspect of the 
Internet is so well-known that it was not 
considered worth mentioning. However, 
those prompted for more details on answers 
such as “the Net is a good source of 
information” did make reference to the 
quantity in follow-up messages. 
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The roles of people using the Internet for 
health included those of being professional 
(nurses, journalists etc.), intermediaries (i.e. 
to research information on behalf of another) 
and patients. Some respondents used the 
Internet in more than one of these roles, 
indicating usage both when ill (i.e. as 
patients) and in good health, out of general 
interest. Using the Internet in terms of 
information seeking naturally depended on 
the role and context of the seeker. The 
following uses were elicited: 

* professional reasons; 

e — self-care, including self-diagnosis; 

* to complement information from a 
medical professional; 

* intermediary usage; 

* peer support (i.e. patient-to-patient); and 

* general interest browsing. 


Professional reasons 

The four health professionals all seemed to 
use the Internet for a large amount of 
information. One said she used the Internet 
“to look at government documents, national 
guidelines, policy documents etc. Also to keep 
in touch with professional nursing 
organisations.” Similarly, “I used the Net for 
searching my local GPs, dentists, hospitals; I 
also ... use specialist Web sites for anatomical 
images, notes on ECG interpretations, some 
specific medical conditions, medical 
procedures, prescription-only drugs ... ” 
Perhaps, unsurprisingly, the health librarian 
respondent uses the Internet extensively for 
professional purposes: “I ... use information 
which is useful and relevant to students and 
staff at the School of Nursing and Midwifery, 
e.g. free e-journals, academic sites, DoH, 
NMAP, NMC, etc. ...” 

One development seems to be the 
signposting by health professionals to their 
patients of Internet resources, despite the 
claim in a recent British Medical Journal 
editorial that some doctors have gone so far as 
to warn their patients, “Whatever you do, 
don’t go on the Internet” (Ferguson, 2002). 
In one case this encouragement was for them 
to “use, in particular, NHS Direct”, for 
self-care and other information. Another 
respondent directs patients to online support 
groups or official sites for various conditions. 
In a follow-up e-mail she added that “often, 
people who use the Net for information would 
not bother to go anywhere else, and it makes 
our job easier if-patients have some 
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knowledge”. This represents an interesting 
development following the well publicised 
phenomenon of patients using the Internet to 
challenge or complement information 
provided by the GP (see, for example, 
Rumbelow, 1999). Of course, the particular 
sample here were Internet-using respondents, 
so there might be a natural propensity 
amongst them to champion the system. 


Self-care 

Self-care was another reason for turning to 
the Internet, including researching symptoms 
and self-diagnosis. This is encouraging, as the 
Government hopes that information provision 
will reduce the number of unnecessary visits 
to the doctor and therefore alleviate the 
burden on the NHS (Wanless, 2002). Eight 
of the respondents mentioned looking for 
information on symptoms, but only four of 
these said this was to avoid visiting a GP or 
medical location: 


I recently found I had shingles and obtained 
advice on self-help — not needing to visit my GP, 
as appointments often take several days to 
arrange. It saves the time of the GP and it is 
easier to ask questions of a PC than a busy 
doctor! 


A health psychology student considered that: 


... Many people do frequently visit their GP for 
minor probs such as colds, when in reality there 
isn’t a lot they can do to help. In these cases I 
suppose that people may feel less inclined to visit 
the doctor if they have access to accurate health 
info from home. 


Another respondent mentioned using self- 
diagnosis flowcharts, as can be found on the 
NHS Direct site, who used sites “that will 
allow me to input symptoms [which] then 
give a list of ‘possible’ causes”. 


To complement information from a medical 
professional 

In addition to self-help, respondents also 
mentioned using the Internet as a 
complementary information service to that of 
the medical professionals. A wide range of 
views was forthcoming; some may be 
considered as neutral, others either positive or 
negative. The “neutral” comments are 
exemplified by a housewife, who said that the 
Internet helps her: 


... understand treatment options and find out 
where to get further information and help (and) 
to carry out research to enable me to get the best 
out of an appointment with a doctor. Before an 
operation or other medical procedure: to 
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understand what is going to happen, and how to 
prepare for it. 


Others made the point that the GP could not 
give as much information as was available 
elsewhere or that there were time constraints 
on the doctor that precluded a comprehensive 
information exchange. One registered 
disabled respondent felt that by using the 
Internet she saved the doctor’s time: 
“appointments are shorter, as I am not 
spending long periods of time asking for 
information”. Another said that she had 
“even printed out material to ask my doctor 
about. And it’s given me questions to ask that 
I would not have thought of before.” The 
issue of how the Internet is helping people in 
their dealings with medical professionals is 
discussed later in this paper. 


Intermediary usage 

Using the Internet in the capacity of 
intermediary — that is, researching things for 
other people — was mentioned by several 
(eight, or nearly 20 per cent) respondents. 
This is a large percentage when one considers 
responses were “open” — there was no 
prompting or options from which to choose. 
A good example comes from a (female) 
company director, who cited no fewer than 
four members of her family, of whom one 
suffered from cancer, another was an 
alcoholic, a third required inoculation and 
other travel advice, and the fourth had bouts 
akin to epilepsy, which the Internet helped 
show the respondent was a misdiagnosis. At 
the end of her e-mail she says, perhaps 
understating the case: “... so I think I can say 
I have a lot of uses and found lots of help/ 
information”. 


Peer support 

Peer support (i.e. patient-to-patient), often 
touted as a great advantage of the Internet, 
was cited by only three of the 42 respondents 
(5 per cent). One mentioned the “ability to 
get in touch with other patients”, and another 
“chatting with other people with the same 
condition”. In a follow-up e-mail the latter 
mentioned specifically Addison’s Disease. 
Self-Help Group. The third respondent was 
the lady with polycythaemia rubra vera. She 
wrote of an e-mail group for sufferers[1] 
“which I also found very useful on practical 
advice, e.g. self-help treatments for symptom 
relief, recommended hospitals, latest 
research”. 
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General interest browsing 

Only five (12 per cent) described activities 
that could be coded as “general interest”, and 
even one of these — a journalist — used the Net 
in this way for professional reasons, to 
research for possible stories. Other 
“browsers” looked for information on 
“general healthy living”, “lifestyle”, or 
“information on any random thing I come 
across, be it news reports, or, well, wherever!” 
The apparent lack of interest shown in 
serendipity by the majority of respondents has 
not been reflected in other findings by the 
authors. A questionnaire on the site of 
SurgeryDoor (Nicholas et al., 2001a, b) found 
that 45 per cent of the 1,068 respondents 
sought advice about keeping fit and healthy, 
and another 20 per cent just came out of pure 
curiosity. Of course, qualitative research 
identifies the various qualities of an activity, 
and does not concern itself with the quantities 
involved. 


Topics sought 

A number of respondents named specific 
topics in which they were interested, which 
they used the Internet to research. There was 
no apparent pattern to these. Several people 
mentioned only one topic: “Alternative 
medicine”. One respondent “would like to 
know what works and what doesn’t but [it is] 
hard to find impartial advice”. Four other 
respondents mentioned the phrase 
“alternative medicine” and three others the 
word “homceopathic”, making a total of eight 
respondents, or nearly 20 per cent. 

A minority of people (five, or 8 per cent) 
said they use the Internet as a kind of 
directory or reference guide — to find, for 
example, medical location addresses, opening 
times, etc. One wrote, “Today my daughter 
wanted to find a dentist that provided NHS 
treatment in our area before she travels.” 
Some went further than this, using the Net to 
“establish the guidelines and parameters of 
NHS treatment”, or to “find out more about 


~the consultant”. A female accountant said 
she: ; 


... was looking at the NHS Direct Web site, as I 
want to be treated at Guy’s and St Thomas’s, 
which is where polycythaemia research is done in 
the UK. I am unsure if the NHS allows me to 
specify which hospital I want and thought this 
Web site would tell-me. 
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The Internet in dealing with the doctor 
Clearly, many of the uses cited above (seeking 
information in addition to that provided by 
doctors, attempting: self-diagnosis, and 
exchanging peer support) may have an effect 
on one’s dealings with medical professionals. 
The questionnaire sought views on the extent 
to which there has been any such impact. 
There was some indication that the Internet 
had enhanced the doctor-patient encounter: 


I visit the surgery less often, often discuss the 
alternatives with the GP, and, on occasion, 
arrive at an agreement with the GP that I will try 
[an] alternative remedy first rather than taking 
an allopathic drug. Sometimes the condition 
disappears with the alternative approach and 
there is no need to see the GP again. 


Another said: 
My GP has been positive about my use of the 
Internet. He freely admits my condition is very 
rare and he has never treated it and he has 
appeared interested in the information I have. 


These are interesting comments, in that they 
reflect the Government’s vision for the future, 
as expressed in the Wanless report, where the 
aspiration is that patients will work in 
partnership with medical professionals. 

There were many comments, on the other 
hand, which suggested that the Internet was 
used as a counter-balance to information (or 
lack of it) provided by the GP, and that there 
was some hostility or disapproval on the part 
of the doctors. 

The most dramatic account was that from a 
38-year-old accountant who has a condition 
named polycythaemia, who said she had: lost 
confidence in the hospital, as my diagnosis 
and treatment have differed on many counts 
from what appear to be the standard 
procedures” and “if the Internet had not been 
available to me, the consultant would have 
made life threatening omissions [sic] in my 
case”. 


Quality and authority of information 

A major issue when considering information 
found on the Internet is, of course, that of 
information quality and accuracy. 


Respondents were asked: “How do you look 


at the issue of ‘information quality’ regarding 
material you find on the Internet? Does it 
make any difference to you from where you 
get the information?” Of course, it needs to be 
stressed that the very act of asking people 
about quality may possibly itself prompt them 
into thinking about an issue about which they 
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may not hitherto have been overly concerned. 
For example, only one of 20 users evaluating 
an Internet health information Web site 
mentioned quality or authenticity issues 
(Williams et al., 2002). The responses on this 
occasion, albeit prompted, show a very real 
concern for the issue. One user said, “quality 
and quantity of information are paramount”. 
Another stated: 
Information quality is obviously important 
especially in relation to health information; it 
could be potentially damaging physiologically 
and psychologically to receive incorrect 
information regarding specific conditions or 
symptoms. 


Unsurprisingly, the medical professionals who 
answered the questionnaire had a good deal 
to say on this subject. One of the nurses said: 


Quality is so important and sometimes one gets 
the impression that the search for “innovation” is 
prized by the DH above everything else. Some 
nursing sites can be misleading when 
innovations are mentioned, but are not 
evaluated, so who knows if they’re any good? 


This is interesting, as it suggests that not even 
so-called “authoritative” sites are always 
correct. This is evidenced in the literature too. 
Coulter et al. (1999), for example, point out 
that even “official” information published by 
the NHS and other government bodies can be 
of dubious quality. 

The free-lance health writer takes a 
different stance. She said: 


I try to choose sites that are recommended by 
people I trust, whether they be professionals or 
friends. ... if someone doesn’t share my 
philosophy or just gives me a bad feeling, I don’t 
really listen. It definitely matters who gives the 
information. 


In other words, this user is imbuing “trusted 
others” with the authority — rather than the 
site producers. Amongst those in the health 
field, however, this view was an exception. 
More common was this, offered by a “health 
professional”: 


I always look for health info from Web sites set 
up by known professional bodies or medical 
schools or colleges, or from NHS and 
government Web sites where info was provided 
by registered health-care professionals. 


The “general public” also showed a great 

concern. over the issue of quality and authority 
(although, as mentioned above, the extent to 
which this was as a result of prompting is not 
clear). A retired 71-year-old summed up what 
appeared to be a general feeling amongst this 
group: “obviously I would tend toward a 
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reputable organisation to provide ... 
information”. Others wrote: “I think it’s 
important to access sites of reputable 
organisations. Some health sites can be out of 
date, biased or misleading”; and “I would 
only rely on information from an accountable 
and authoritative source such as the NHS, 
company Web sites, professional or academic 
institutions”. In total, 12 of 33 replies (36 per 
cent) to this question indicated a preference 
for accessing information from “official”, 
“reputable” or “well-known” sites. One 
person cautioned, however, that “any 
information that you are unsure about should 
be checked with your GP/Dr and should not 
be the basis for important decisions”. 

Of course, perceptions of quality and 
authority often depend on the viewpoint and 
beliefs of the information seeker (Nicholas, 
2000). Of interest to the present writers are 
people who take or approve of so-called 
“alternative” medicine, as they may regard 
the NHS as not being an authoritative voice 
when it comes to this kind of treatment. 
Typical of this group was the lady who said: 


Mostly I go to the Net to find alternative 
medicine approaches to dealing with issues of 
health care, especially homeopathic remedies, 
ayurvedic practices and natural or herbal 
remedies. I also practise yoga, so I will often 
check out those sites for possible solutions to 
problems. 


In a questionnaire survey posted on the 
SurgeryDoor Web site (Nicholas et al., 2001a, 
b) we found alternative medicine users were 
more likely to use information found on the 
Net to replace a visit to the doctor, perhaps 
indicating a disenchantment with orthodox 
remedies and perhaps even with the NHS 
itself. Indeed, work undertaken in evaluating 
health initiatives on digital interactive TV 
indicated that those interested in alternative 
medicine were less likely to trust the NHS 
than other groups (Nicholas et al., 2003). 

There was some indicative evidence of this 
phenomenon, although it was not a prevalent 
theme, as, of course, the recruitees by 
definition all used the NHS site. 
Nevertheless, one respondent claimed, “I 
guess I never just trust what the NHS says”, 
having offered her view that: 


Western medicine in general is myopic in its 
approach to the matter of health ... there is too 
much of the notion that current medical 
practices can “heal” you and too little on the 
concept of responsibility for one’s own state of 
health ... What the NHS offers is just one part of 
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the whole. ... I always research their diagnosis as 
well as their proposed remedy before using it. 


Tactics for validating information included 
checking the organisation responsible, in one 
case to see if it is “transparent and not hiding 
anything about their credentials or the sources 
of the info they provide”, and in another to 
see “whether it is a recognisable source”. Of 
course, for those interested in alternative 
remedies, particular kinds of orthodox 
treatments, or in specific conditions these 
“recognisable sources” will be different. 
Further work is required to delve into this 
topic at a deeper level. Regarding the present 
sample, one person said he asked himself: 


Is it complete, factual, provable, first-hand and 
truthful? Is it up to date, current and from a 
reliable source? ... Should I equate the 
infomation [sic] gleaned in UseNet with that 
obtained from NHS Direct? No, but with some 
understanding of those sources a judgement can 
be made and inferences drawn, 


There was an indication from some that 
Internet-hosted information was unregulated 
and emanated from a vast number of different 
sources: “all information on the Net is 
uncensored and could have been tampered 
with”. 

Trust was an issue that came out in the 
replies to questions on authority and quality 
and, in one case, in reply to a question on sites 
visited. Six respondents (14 per cent) gave 
answers that were coded as being related to 
“trust”. These people were generally 
supportive of the NHS. One respondent, a 
35-year-old housewife, said: 


In terms of published information, I have a 
positive view of the NHS (i.e. I trust the 
information they publish). In other areas, such as 
trusting my health or that of my family to the 
NHS, I have a balanced view based on realistic 
expectations and personal experience (both good 
and bad). 


Others mentioned trusting the NHS “to a 
large extent” or “on the whole”, but “also 
look elsewhere for the information”. This 
echoes a constant theme of Internet usage — 
that people now will look at several Internet 
sites (and, therefore, consult several 
organisations) for information. This appears 
to be true even when people “trust” one site, 
or when they go to predefined site types, such 
as “academic” etc. The ease of information 
access seems to have made Internet users 
connoisseurs. 
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Another issue was raised regarding trust. 
One respondent considered that the 
information was: 


... trustworthy in the sense that the info present 
will be accurate on the whole 


but went on to say: 
but biased in the sense that there is an economic 
consideration with health-care provision and the 
NHS is representing the DoH policies and may 
not promote treatments that are not available 
widely on the NHS. 


Conclusions 


This paper has reported on a study which, 
unusually, attempted to capture qualitative 
data from a questionnaire, which, 
accordingly, used only open questions. A total 
of 42 people responded to the survey, 
providing over 50 pages of observations, 
opinions, and rich accounts of practices and 
the motivations behind them. Respondents 
were forthright and open, reflecting findings 
from previous studies, which found that 
online communication yielded substantial and 
high quality data. Only a disinclination to 
engage in more than one communication 
exchange disappointed the researchers. It may 
be that too much information was sought in 
the initial questionnaire, and hence 
respondents felt they had little more to offer. 
Social science researchers have indicated the 
long time required to gradually establish 
rapport and trust with research subjects. 

With regard to the survey results, the survey 
showed that the Internet is exploited, in a 
wide variety of ways, by information seekers 
acting in a number of roles. Often the same 
individual uses the system in different 
capacities — as a patient, an intermediary or a 
professional. 

Indicative evidence is offered that some 
health professionals are now embracing the 
Internet to the extent that they direct patients 
to appropriate Web sites such as those of 
support organisations and the NHS. Lay 
users showed a high level of understanding of 
issues such as quality and authority. In some 
cases this manifested itself in an interesting 
(and healthy?) scepticism towards the NHS 
and its information agenda. 

Overall, the survey was a success in terms of 
its methodology and in the amount and 
richness of the data accrued. As always with 
qualitative research, more questions are posed 
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from the results than are answered. Follow-up 
work will explore further issues such as 
authority (and “trusted others”) and the 
complex relationship between patient, 
information and medical professional. 


Note 


1 http://members.aol.com/mpdsupportfindex.html 
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It has been argued that people may be more willing to 
seek potentially threatening information on the Internet 
than they would in “real life”. For alcohol drinkers, 
potentially threatening information includes diagnostic 
information on the amount they drink, and information 
about the negative effects of alcohol consumption. In the 
present study, participants with varying levels of alcohol 
intake and plans for change chose four alcohol-related 
articles to read (from 12), using either the World Wide 
Web (WWW) or pen and paper. Results showed that 
drinkers not currently reducing their drinking were more 
likely to seek diagnostic, potentially threatening anti- 
drinking information via the WWW compared with when 
seeking paper-based information. Drinkers either 
contemplating or engaging in efforts to reduce their 
drinking sought pro-drinking information on the WWW 
and anti-drinking information when using pen and paper. 
The potential role of the Internet, and perceived 
anonymity, in health promotion are discussed. 
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Introduction 


According to a recent survey of Internet users 
in the USA (Pew Internet and American Life, 
2002), 63 per cent of Internet users have used 
the Internet to research health information — 
and in any one day six million US Internet 
users access health information online. The 
most prevalent use of the Internet is to 
research a specific illness (93 per cent of 
“health seekers”), followed by information 
aimed at weight control and nutrition (65 per 
cent). When questioned about the impact of 
their information seeking, 34 per cent of 
“health seekers” said that they had changed 
their approach to maintaining their own or 
another’s health, and 30 per cent said that 
they had changed the way they think about 
their diet, exercise or stress. This pattern of 
responses matches those found by Nicholas et 
al. (2001) in a survey of users of a health 
information Web site — 47 per cent of users 
said that the site had helped with their 
dealings with their doctor, and fully 58 per 
cent of users currently suffering from a 
condition stated that the Web site had helped 
to improve their condition. 

The use of the Internet for health 
information is not limited to accessing 
information. There are large numbers of 
electronic support groups (ESGs) available 
online Joinson, 2003). A number of 
empirical studies have examined, for instance, 
the use of the Internet by people diagnosed 
with sports injuries (Preece, 1999), eating 
disorders (Winzelberg, 1997) and depression 
(Galegher et al., 1998). These studies suggest 
that the main features of online social support 
are that it reduces feelings of isolation and 
provides information and emotional support 
(Preece, 1999). 

There are a number of factors that may well 
unite users of online ESGs and health 
information seekers (indeed, much of the time 
they may well be the same people). For 
instance, a proportion of health information 
seekers have reasonably well developed 
strategies for evaluating the quality of 
information on health Web sites (Pew 
Internet and American Life, 2002), while 
users of online social support use a variety of 
linguistic devices to establish legitimacy and 
authority in online forums (Galegher et al., 
1998). Ease and flexibility of access are 
another possible link: some studies of ESGs 
have suggested that most use occurs outside 
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office hours, when traditional forms of 
medical and social support are not available 
(Winzelberg, 1997). A further motivation 
(and potential barrier) is the users’ perceived 
anonymity when online. For instance, the 
Pew Internet and American Life (2002) 
report notes that 80 per cent of health seeking 
occurs from a home connection. One 
participant in a follow-up focus group is 
quoted as saying, “If I did work, I would 
probably be too embarrassed to look up stuff 
online.” Another focus group participant 
echoed her concern, saying, “I know we 
monitor the use of the Net at work and I have 
seen it work against people.” These quotes 
suggest that one reason for seeking health 
information online is the relative anonymity 
and convenience afforded by home use 
(Borzekowski and Rickert, 2001). Nicholas 
et al, (2003) have developed the concept of 
“search disclosure” to help explain why 
people moderate their search strategy 
dependent on the perceived anonymity of the 
search location. Looking at the data from 11 
different studies, Nicholas et al. found that 
people search for information on sensitive 
topics during “private” times of day, and at 
information kiosks located in places that 
afford more privacy. 

This effect of anonymity on health 
behaviour is not just limited to the searching 
for health information. For instance, 
Davidson et al. (2000) found that, for both 
face-to-face and online social support 
groups, the number of support groups 
available positively correlated with the 
social burden imposed by the ailment, 
particularly the embarrassment associated 
with a particular diagnosis or treatment. 
Similarly, Mickelson (1997) found that 
users of a “parent support” ESG reported a 
greater perceived stigma associated with 
their child’s diagnosis, greater visibility, and 
lower expectations of support from their 
parents and casual friends, compared with 
non-ESG users. Joinson and Banyard 
(2002, Study 2) found that information 
access on a cancer Web site was more 
closely linked to prevalence data than 
queries to a telephone helpline. In the case 
of stigmatised cancers, this effect was 
particularly pronounced. Barak (2001) 
notes that the anonymity afforded by the 
Internet.has seen high levels of usage on his 
suicide prevention site by adolescents, shy 
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and socially anxious people, and those with 
a high public profile. . 

Although Internet users seeking diagnostic 
clinical information on the WWW or support 
via ESGs have been reasonably well 
researched, the seeking of health promotion 
information on the Internet is something of an 
unknown. One possible prediction is that the 
information-seeking behaviour of people 
either unaware of a risk or not contemplating 
changing an unhealthy behaviour is unlikely 
to be highly motivated (Prochaska et al., 
1992). For instance, a study of a staff health 
promotion site showed that users were 
predominantly those already exercising, 
rather than those either not taking any 
exercise or those contemplating taking more 
exercise than at present (Griffin et al., 2000). 
Indeed, a large body of social psychological 
research suggests that people will go to great 
lengths to avoid potentially threatening 
(health) information (Taylor, 1989). This is 
particularly the case if health information is 
seen as personally relevant (Sherman et al., 
2000) or if it would cause anxiety by raising 
the possibility of future illness (Brashers, 
2001), or presumably if there were privacy 
concerns linked to the seeking of the 
information (Nicholas et al., 2003). 

The aim of the present study is to compare 
health promotion information seeking on the 
WWW and paper for people with varying 
degrees of unhealthy behaviour, and at 
different stages of change. The health 
behaviour, and related health promotion 
information seeking, considered in the 
present study are alcohol intake. There is 
considerable evidence that high levels of 
alcohol intake by university students can have 
negative consequences (Migneault et al., 
1999). In the present study, college students’ 
information seeking of pro- and anti-drinking 
information is examined in the light of their 
weekly alcohol unit intake, and their plans for 
behaviour change. It is hypothesised that the 
information-seeking strategies of individuals 
using the Internet will differ from those of 
participants using “traditional” information- 
seeking methods. The form this difference 
takes is predicted to interact with individual 
differences, specifically the individual’s plans 
for drinking reduction, and their current unit 
intake. 
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Method 


Design 

A three-way, between-subjects design was 
used in the present experiment. Participants 
completed the measures using either the 
WWW or pen and paper, and were 
categorised according to their unit intake of 
alcohol (high vs low) and their plans to reduce 
their drinking (not contemplating or 
contemplating/engaging in change). 
Participants’ requests for articles to read 
formed the information-seeking 
dependent-variable. 


Participants 

Participants were 117 undergraduate students 
at two UK universities (25 males, mean 

ages = 20.74 years (SD = 4.82) and 21.72 
(SD = 4.30) for each location). The eight 
participants who abstained from alcohol were 
not included in subsequent data analysis. 
Participants were allocated to either the paper 
version of the study (n = 73) or a WWW 
version (n = 44). 


Materials 

Participants completed two measures before 
the information-seeking part of the 
experiment. 


Demographic variables 

Participants were asked to enter their age, 
gender and alcohol intake (options: “do not 
drink at all”, “1-5 units”, “6-10 units”, 
“11-15 units”, “16-20 units”, “21-25 units”, 
“26-30 units” and “31+ units”). It was 
explained in the text that “1 unit = half a pint 
of beer, one spirit measure, one small glass of 


» 


wine”. 


Stage of change measure 

The stage of change measure followed the 
procedure developed by Prochaska and 
colleagues (1992). Participants were asked to 
“please mark the statement that best sums up 
your current state”. The four statements (“I 
do not intend to reduce my drinking within 
the next six months”, “I intend to reduce my 
drinking within the next six months”, “I 
intend to reduce my drinking within the next 
month, and have begun taking small steps”, 
and “I am actually reducing my drinking right 
now”) correspond to the pre-contemplation, 
contemplation, preparation or action stages of 
change respectively. 
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Information seeking 

Once participants had completed the health 
measures, they were presented with a choice 
of 12 articles for evaluation. For each article, 
a title and brief summary were provided, 
along with a box in a column headed “click 
the box to evaluate this article”. The articles 
represented six different types of alcohol- 
related information (two each). 


Pro-drinking articles 

Informative 

Sources that give information about the 
positive physical, social and emotional 
consequences of alcohol consumption. For 
example, “Drink to be Healthy. It’s official! A 
report into the reduced risk of heart disease in 
people who drink moderate amounts of red 
wine.” 


Promotional 

Messages from commercial sources 
promoting the purchase and consumption of 
alcohol. For example, “What drink to buy, 
and where to buy it: a guide to the wide range 
of drinks on the market with details about 
their alcohol content, taste and general 
quality. The source also includes information 
about the cheapest places to purchase your 
drinks.” 


Behavioural 

Sources of information on activities to be 
carried out with alcohol. For example, 
“Drinking games: bored of the same old 
conversation in the pub? Try some of these 
drinking games that help the evening move 
along and liven up even the dullest friends.” 


Anti-drinking articles 

Informative 

Sources that inform about the level of 
individual risk, and possible health 
consequences — allow the individual to | 
calculate risk. For example, “Home is where 
the heart is, but the bar is for heart attacks: 
information about the effects of drinking on 
the heart and how to estimate your own risk of 
heart problems.” 


Promotional 

Sources that seek to encourage abstinence or 
restrained alcohol use. For example, “Alcohol 
and the accident-prone: information about 
how your chances of having an accident at 
home, at work or in the car can be affected by 
even small amounts of alcohol”. 
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Behavioural 

Practical advice on how to limit personal 
consumption and control excessive drinking. 
For example, “The Tony Adams Story: the 
England and Arsenal footballer describes the 
techniques he uses to keep off alcohol.” 


Procedure 

Participants were approached during a 
training session in Internet use. If they agreed 
to take part, it was explained that the study 
was examining the evaluation of articles about 
drinking. In the WWW sample, participants 
were then given the World Wide Web address 
of the questionnaire, and asked to work 
through the questions. Once the pre- 
measures had been completed and 
electronically submitted, participants were 
automatically shown the list of potential 
articles for evaluation. Written instructions 
informed participants that: 


Listed below are 12 articles about drinking and a 
short summary of each article. Choose four to 
read and evaluate by clicking the box next to 
your chosen article. When you have selected the 
four articles, click the submit button at the 
bottom of this page. The articles will then 
automatically be sent to your Web browser for 
evaluation by you. 


In the paper condition, the instructions were 
the same except that any Internet references 
were changed to instruct participants to “tick” 
the boxes, and hand the completed responses 
to an experimenter for collection of the 
articles to evaluate. 

Once subjects had chosen their articles, it 
was explained that the aim of the experiment 
was the act of choosing, not the evaluation, 
and that articles would not be forwarded for 
evaluation. In the WWW condition, a series of 
links to similar articles on the Internet were 
provided for participants to follow if they 
wished. In the paper condition, these links 
were provided in a print-out. Participants 
were thanked for their help, and any questions 
answered by the experimenters. 


Results 


Initial analyses were conducted to investigate 
differences between the seeking of either anti- 
or pro-drinking information. Because seeking 
one type.of information reduced the 
likelihood of seeking the other, an 
information-seeking index was calculated by 
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subtracting the number of anti-drinking 
articles chosen from the number of pro- 
drinking articles chosen. Thus, if a participant 
wanted to read four anti-drinking articles, and 
no pro-drinking articles, their index score 
would be -4. If they sought two anti- and two 
pro-drinking articles, their index score would 
be 0. The mean index score was 0.59 

(SD = 2.38), indicating a slight bias toward 
pro-drinking articles. Participants’ scores on 
the information-seeking index did not differ 
according to their university location 

(F (1, 114) = 0.05, p > 0.82) or gender (F (1, 
114) = 0.42, p > 0.5). Location and gender 
were therefore not included in any subsequent 
analyses. 

Because any effect of the WWW on 
information seeking might interact with 
individual differences, participants were 
categorised using two nominal measures from 
the questionnaire. First, their self-reported 
alcohol intake was categorised as either high 
(above and including 11-15 for a female, and 
above and including 16-20 for a male) or low 
(below 11-15 for females, 16-20 for males). A 
second category was created by collapsing 
participants’ scores on the Stage of Change 
measure into “not contemplating change” 
and “contemplating or engaging in change”. 
A three-way, between-subjects ANOVA 
(medium (WWW vs paper) x unit intake 
(high vs low) x stage of change (not 
contemplating vs contemplating or engaging 
in change)) was calculated to examine the 
interaction between individual differences 
and information seeking. 

Medium (F (1, 101) = 4.89, p = 0.03, Ms 
1.05 and 0.53 for WWW and paper 
respectively) and unit intake (F (1, 101) = 
7.03, p < 0.01, Ms 0.08 and 1.25 for low and 
high unit intake respectively), but not stage of 
change (F (1, 101) = 0.12, p > 0.73, ns, Ms 
0.81 and 0.56 for pre-contemplation and 
contemplation or engaging in change 
respectively), showed significant main effects 
on information seeking. To provide a frame of 
reference to these results, effect sizes were 
calculated (Cohen, 1977). As a guide, Cohen 
proposed that a “small” effect is 0.20; a 
“medium” effect is 0.50 and a “large” effect is 
0.80 (Cohen, 1977), Using Cohen’s 
standardised mean difference (d,), the effect 
sizes are 0.22 for medium, 0.49 for'unit intake 
and 0.11 for stage of change. 

The interaction between medium and 
unit intake was marginally significant 
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(F (1, 101) = 3.35 p = 0.07). This interaction 
is illustrated in Figure 1. 

The interaction between medium and stage 
of change was significant (F (1, 101) = 7.85, 
p < 0.01). This interaction is illustrated in 
Figure 2. The interaction between unit intake 
and stage of change was not significant 
(F < 0.1). The interaction between medium, 
unit intake and stage of change was also 
non-significant (F < 2). 


Discussion 


Participants using the WWW to seek alcohol- 
related articles sought significantly more pro- 
drinking information than those using pen 
and paper. This confirms the hypothesis that 
people would seek different types of health 
information online compared with pen and 
paper. In line with earlier work on health 
behaviour, those with a high alcohol intake 
sought significantly more pro-drinking articles 
than those with a low unit intake. There was 
no main effect for stage of change on 
information seeking. 

The interaction between medium (WWW 
vs paper) and unit intake (high vs low) was 
marginally significant. When unit intake was 
high there was no difference in information- 
seeking bias between WWW and paper 
conditions. However, when unit intake was 
low, participants answering using paper 
sought primarily anti-drinking articles, while 
participants using the WWW sought primarily 
pro-drinking articles. 

The interaction between medium and stage 
of change was significant. Specifically, when 
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participants were not considering reducing 
their drinking (pre-contemplation), they 
sought primarily pro-drinking information, 
with those using the WWW showing less of a 
preference for pro-drinking behaviour than 
those using paper. Individuals who were 
either contemplating or engaging in drinking 
reduction tended to seek pro-drinking 
information on the WWW, and anti-drinking 
information on paper (see Figure 2). It would 
seem, therefore, that on the WWW people are 
acting in a manner contrary to expectations: 
those planning or engaging in drinking 
reduction seek pro-drinking information, 
while those not planning to reduce their 
drinking seek a balance of pro- and anti- 
drinking information. On paper the pattern of 
information seeking is as would be expected: 
those engaging in change sought anti-drinking 
information, while those not contemplating 
change sought pro-drinking information. 
This pattern of responses might represent a 
socially desirable response on paper 
compared with on the WWW. In the present 
study, the paper requests were handed to the 
experimenter — in both locations the 
participants’ lecturer. Although procedures 
were set in place to protect anonymity of 
participants, the seeking of information on 
paper was more public, and less anonymous, 
than the WWW condition. It would be 
expected that increases in the “publicness” of 
a behaviour would increase socially desirable 
behaviour (Paulhus, 2002), a pattern already 
established in Web-based surveys (Joinson, 
1999) and observed in the use of health 
information kiosks and health Web sites 


Figure 2 The interaction between stage of change and medium on 
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(Nicholas et al., 2003). The confounding of 
medium and levels of anonymity (and hence 
social desirability) make the interpretation of 
any cross-medium results difficult to 
interpret, and would need to be addressed in 
any further studies (for instance, by increasing 
identifiability in the Web condition, or 
reducing it in the paper condition). However, 
the pattern of results from the less private 
paper condition is consistent with the notion 
of “search disclosure” (Nicholas et al., 2003). 
It would be interesting to isolate the unique 
effects (if there are any) of medium and level 
of anonymity in health information seeking. 

Looking at the results from the WWW 
alone, this study suggests that health 
information seeking on the Internet is a 
double-edged sword. While the WWW 
delivery mechanism encouraged those not 
planning to reduce their drinking to seek 
articles with a slightly less positive slant than 
average, this effect was reversed for those 
planning or engaging in drinking reduction. 
The behaviour of those currently engaging in 
change is unusual — that people should seek 
predominantly pro-drinking messages, while 
either contemplating or engaging in drinking 
reduction, is somewhat counter-intuitive. It 
may be that many of those classified within 
this group may have been contemplating 
change, a stage associated with traditional 
“social exchange” processes (i.e. the weighing 
up of pro and anti arguments and outcomes) 
(Janis and Mann, 1977). 


Implications for health promotion on the 
Internet 

Most studies of health information seeking on 
the Internet have focused on access to sites 
providing medical information for those 
diagnosed with a condition, or their carers 
(Joinson and Banyard, 2002; Nicholas et al., 
2001; Pew Internet and American Life, 
2002). The results of the present study 
suggest that the seeking of health promotion 
information may be motivated in a different 
way. First, there seems to be an effect of 
medium — if traditional modes of information 
seeking have a level of identifiability attached, 
people seem to behave in a socially desirable 
manner (i.e. seeking somewhat anti-drinking 
information). Second, the current health 
behaviour and plans for change lead to 
different information-seeking motivations, 
which may well also interact with medium. In 
the present study, these interactions led the 
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pattern of information seeking on the Web to 
appear somewhat counter-intuitive and 
contrary to expectations. Further research 
with a larger, more varied sample and tighter 
control of anonymity and privacy is needed to 
identify the exact nature and causes of these 
interactions. 

© A note of caution must be sounded in the 
interpretation of the results. In the present 
research, the Internet sample consisted of 
students assigned to the WWW condition. 
This was necessary to allow post hoc contrasts 
with a comparable group using paper 
measures. However, this design is not a true 
representation of the environment in which 
most people access information on the 
Internet, nor does it replicate possible 
differences between an Internet and paper 
sample. Indeed, it could be argued that any 
effects might be greater, if the sample 
consisted of a “genuine” Internet sample 
rather than an allocated sample because of the 
increased privacy afforded to Internet users 
outside the laboratory. 

The present research shows that the 
Internet may prove to be an unrivalled 
environment for the provision of health 
promotion information aimed at encouraging 
people to change unhealthy behaviours. 
However, the WWW may also have 
previously unforeseen consequences in the 
seeking of health-related information amongst 
those contemplating, preparing or engaging in 
health-behaviour change. Continued research 
effort that investigates how users browse 
health promotion information on the WWW 
is urgently required. The current trend is for 
health promotion professionals to simply 
convert existing paper resources to digital 
format. Although this approach is a 
commendable effort toward providing access 
to health promotion material, we would 
suggest that researchers need to know who is 
likely to seek specific information using the 
WWW, and should tailor their resources to 
meet the motives of those users and their 
perceived anonymity. 
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Introduction 


Touch-screen technology has been used for a 
wide range of interactive or non-interactive 
applications. An annual world-wide increase 
of 25 per cent in the use of touch-screens was 
predicted in 1996 (Cranston et al., 1996). 
Five groups of touch-screens were identified: 
point-of-sale, information kiosks, industrial, 
gaming and business applications. Of these, 
the business and education markets were 
slowly picking up the technology, while 
point-of-sale and information kiosks were 
adopting touch-screen technology at a very 
rapid pace. 

A number of companies specialise in 
information touch-screens and interactive 
solutions or interactive public access 
terminals or kiosks in the UK. Some of them 
collaborate with the public sector, national 
institutions and local authorities for the use of 
touch-screens by the general public in a host 
of areas (Neoproducts, 2003a; Technology 
Active Solutions, 2003a; InTouch with 
Health, 2002). 

The availability and the use of touch-screen 
information systems for accessing a range of 
information in different settings in the UK 
were examined. Settings included 
employment services, community services 
and health. Availability and use of 
touch-screens from socially excluded or 
hard-to-reach groups were also examined in 
particular. 


Methodology 


Literature review 

Academic, “grey” and commercial literature 
was explored. Academics with extensive 
experience in the field were contacted and 
relevant literature was requested. Other 
published and unpublished literature was also 
obtained using Internet searches. Company 
Web sites were explored. Extensive Internet 
searches were conducted using the general 
term “interactive technology” and the terms 


2 6. 


“touch-screen kiosks”, “touch-screens & 
» eC. 


social care”, “touch-screens & community 
services” and “touch-screens & health”. 


' Consultation with stakeholders (other 


than users) ` 
A total of 13 consultations with stakeholders 
were conducted in January 2003. These were: 
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* academics or others who researched or 
worked in the field; 

* representatives of touch-screen 
companies (manufacturers, designers and 
developers of health-related touch-screen 
systems); and 

* others who offer information and choice 
using touch-screens, i.e. representatives 
of NHS Direct (NHS Direct touch- 
screen kiosks), the Department for Work 
and Pensions (Jobcentre Plus Jobpoint 
kiosks). 


Meetings took place in times and venues 
convenient to the stakeholders. E-mail and 
telephone consultations were used either in 
addition to face-to-face meetings or as an 
alternative method when other commitments 
did not allow time for a face-to-face meeting. 
Informed consent was sought from all 
stakeholders when they were invited to take 
part. Consent to tape-record the 
consultations was sought and was given in 
most cases. A professional tape-recorder was 
used, with a telephone connector attached for 
the telephone consultations. In some cases, 
e.g. consultation with a Hindu Priest in a 
Hindu Temple, the use of the tape-recorder 
was considered inappropriate by the 
researcher and a manual record was taken. 


Combining literature review and 

consultation with stakeholders 

The literature review was conducted in 

parallel with the consultations with 

stakeholders. The consultations shed light on 
formal and informal literature and some 
literature led to particular companies and 
projects and assisted in the identification of 
stakeholders. The two supplemented each 
other; thus the findings are presented 
together. It was generally assumed that touch- 
screen technology facilitates social inclusion; 
however, reference is made to projects which 
target specific socially excluded groups. The 
findings are themed under four general 
headings: 

(1) Availability and use of touch-screens for 
employment. 

(2) Availability and use of touch-screens for 
community services (and targeting 
socially excluded groups). 

(3) Availability and use of touch-screens for 
health information and promotion. 

(4) Availability and use of touch-screens with 
health information and promotion 
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content targeting socially excluded 
groups. 


Availability and use of information 
touch-screens in the UK 


Availability and use of touch-screens for 
employment l 
The most widely available information kiosks 
are the Jobpoint kiosks provided by the 
Department of Works and Pensions. The 
decision to move on to touch-screen 
technology was taken by the Department as 
part of a review of all business processes 
(Browes, 2002). The kiosks host the second 
largest jobs database in the world. A total of 
9,000 kiosks have been available in 1,000 UK 
Jobcentres, providing access to over 500,000 
jobs, including European, international and 
third-party, since February 2002 
(Neoproducts, 2003b; Browes, 2002). They 
were produced by Neoproducts and won 
many awards (Neoproducts, 2003b). Active 
user involvement was achieved throughout, 
through consultations with real users 
(Browes, 2002). Accessibility, human factors 
and special needs were fully incorporated by 
using occupational psychologists specialising 
in human-computer interaction and 
specialists in accessibility issues 
(Neoproducts, 2003b; Browes, 2002). 


Pilots and evaluation in Jobcentre settings 
Piloting of kiosks in Jobcentre settings began 
in 1996 (two Jobcentres), and in 1998 (ten 
Jobcentres) — Open Access Pilot (Browes, 
2002). The early kiosks were accessed by 
swipe card and had integrated telephones and 
print-out facilities. The first pilots were 
thoroughly monitored and evaluated. The 
early evaluations in 16 Jobcentres showed 
high satisfaction by users and staff 
(Department of Work and Pensions, 2001). 
The swipe cards and the integrated 
telephones were, however, abandoned, as the 
cards were considered costly and the 
telephones unpopular. Currently freephone 
access to Jobseeker Direct is available from 
separate units. 


Pilots and evaluation in non-Fobcentre settings 
In 1999, two kiosks were piloted in a non- 
Jobcentre plus location — Local Access Pilot. 
They were situated in a community learning 
opportunity centre in a small ex-mining 
village in North East Derbyshire, Cillamash. 
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The kiosks displayed jobs and four Web sites 
with job-related information and also had 
integrated telephones (Browes, 2002). 
Evaluation of this pilot showed that users 
were highly satisiied. A Jobpoint kiosk is still 
available there. 

Remote Jobpoints were evaluated in two 
Tesco supermarkets in 2002 (GHK & PRI of 
Leeds Metropolitan University, 2002). The 
kiosks offered access to job vacancies for a 
range of clients that generally did not 
constitute a large proportion of Jobcentre 
users, for example, young people and women. 
Although user satisfaction was high, fewer 
people using the remote Jobpoints selected 
jobs for which to apply compared with 
Jobcentre users of Jobpoints. The main issue 
raised by staff was the reliability of the 
machines and the need to minimise the 
amount of downtime. 


Further pilots and pians 

Phase 1 pilots include rolling out external 

Jobpoints in (Browes, 2002): 

* HM Prison Service: currently available in 
four prisons and aimed at low risk 
prisoners preparing for release. 

e Retail stores: about to extend to a further 
ten Tesco stores and one Asda/WalMart 
store. 

e Stansted Airport: currently in the process 
of placing two touch-screens. 

* Community learning centres: one already in 
place and another one planned. 

* Jobcentre Plus “satellite” sites: currently in 
the process of rolling those out in small 
job centres, which only have job 
vacancies. 


Usage statistics 

On average each kiosk gets 350 page hits per 
day, or 35 job hits per kiosk per day. Monthly 
each kiosk gets 63 million page hits or 63 
million job hits January 2003). Importantly, 
80 per cent of all the hits on the Jobcentre 
Plus Web site are coming from the Jobcentre 
kiosks, not from personal PCs with online 
access (Neoproducts, 2003b). 


Users’ satisfaction 

Users, in general, were satisfied in 2002 

(Browes, 2002): 

* 81 per cent of users were satisfied with 
Jobpoints; 

* 92 per cent found Jobpoints easy to use; 
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‘48 per cent reported an increase in 


satisfaction with Jobcentre Plus services; 
and l 
* between 31 per cent and 48 per cent 

reported an increase in number/types of 
jobs looked at, geographical area 
searched, number of applications made, 
knowledge of types of jobs on offer and 
their confidence in getting a job. 


Other benefits of the system 

These kiosks provide access to socially and 
technologically excluded groups. They 
empower users, offer access to jobs at local, 
national and international level and allow 
access in remote environments. According to 
the consultations, their use also frees up a lot 
of staff time and allows staff to deal with those 
who have difficulties or those with special 
needs, such as those who cannot stand, those 
who have literacy or numeracy difficulties, do 
not understand English etc. The Jobpoints 
also transform the offices aesthetically. 


Limitations of the system 

In the open access pilot of the Jobpoints, 
sitting-down touch-screen kiosks were 
available. However, it was found that the first 


_ people in during the morning were seating 


themselves in the place for the day. Since the 
system needs to cater for a large and transient 
population, sitting-down arrangements were 
abandoned. However, this may present 
problems in catering for those who have 
difficulty standing. 

Although the system is available in the 
Welsh language at Welsh offices, there are 
currently no plans to make it available in 
other languages. According to the 
consultations, this would involve a substantial 
cost element, which may not solve access 
issues for non-English speakers, as many 
people from ethnic minority groups, 
particularly from the Asian sub-continent, are 
also illiterate in their own language. On a 
practical level, it would be impossible to 
translate all vacancies into other languages. 


Future developments 

Future developments under consideration 
include additional Jobpoint functionality, i.e. 
providing information to support job seeking, 
development of applications to enable display 
on third-party kiosks (e.g. in the Sheffield 
City kiosk), and wireless/Internet delivery of 
services to support staff and clients outside 
the Jobcentre network. 
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Availability and use of touch-screens for 

community services (and targeting 

socially excluded groups) 

Providing accessible electronic information 

about community services, social support and 

local services to people who do not have 

access to appropriate technology at home or 

who lack knowledge about how to access 

electronic information in itself facilitates 

social inclusion. Some examples of touch- 

screens aiming to provide information to the 

general public and located in public places are 

presented. Some of these systems also provide 

health information; some other systems have 

specific channels that target specific socially 

excluded groups. These are: 

»  J-plus information points; 

* touch-screen kiosks in deprived and rural 
areas; 

* touch-screen kiosks for older people; 

* kiosks aiming to facilitate social 
democracy; and 

* Choice Bedding Letting for Brighton and 
Hove Council. 


I-plus information points 
J-plus information touch-screen points are 
available in more than 150 UK locations 
(Cityspace, 2003). Cityspace operates the 
points in partnership with City Local 
Authorities and Municipalities; they allow 
them to align with the Government’s 
Modernising Agenda. They are usually 
located on high streets, in bus hubs and local 
shopping centres. They provide easy local 
access to digital services such as e-mail, links 
to local authority Web sites and booking and 
ticketing facilities. Users are able to read the 
news, look at tourist and online travel 
information, as well as informing the council 
about issues such as graffiti, abandoned 
vehicles, pot-holes, and other “street 
problems” (Cityspace, 2001a, b). Recent 
additions include a Public Journey Planner 
(Cityspace, 2002a). Future online 
applications will include facilities for council 
tax payment, parking fines, e-voting, e-forms, 
and information about benefits entitlement. 
I-plus kiosks are currently available in many 
London Boroughs (Cityspace, 2002b, c, 
2003). There are also networks in Bath, 
Bristol, Manchester, Reading and Knowsley — 
Knowsley was the first location out of London 
(Cityspace, 2001c, 2002d; Knowsley, 2000). 
Some channels target specific groups. 
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I-plus points for those who suffer from domestic 
violence. This channel, currently available in 
most kiosks, acts as a digital signpost to 
information and advice for those in violent 
and abusive relationships (BBC News, 2002). 
Being located near transport centres and 
transport hubs, the points provide the perfect 
opportunity for people to look at the channel 
in a safe environment where their identity is 
protected (Cityspace, 2002e, f). This channel 
has proved particularly popular with users. In 
less than a month from its nation-wide launch 
(March 2002), nearly 5,000 women sought 
advice and information on the issue of 
domestic violence. In its first month, an 
average of six women per day per kiosk 
accessed the channel. 

I-Plus for homeless people in London. The 
Web site: www.homelesslondon.org.uk was 
launched in June 2002 (Innovations, 2002; 
Cityspace, 2002g; Payne, 2002). It has been 
compiled and maintained by the Resource 
Information Service. It is currently available 
on 70 Cityspace I-plus touch-screen kiosks 
located on London’s streets (a number which 
is set to rise to 150) as well as through 
personal computers in day centres and 
hostels. In future it will be available through 
3G Internet services accessible by mobile 
phones. The kiosks have a print-out facility 
and a free e-mail service, which enables users 
to send an e-mail, anonymously, to reassure 
relatives and friends of their safety. This 
initiative, which forms part of the Lord Mayor 
of London’s Strategy on Homelessness (the 
Rough Sleepers Strategy), is the first of its 
kind. The site contains details of over 3,000 
services, including details of accommodation, 
employment and specialist support. It allows 
users to identify and locate their nearest 
hostel, find out information about 
rehabilitation centres and community 
services, helplines and counselling facilities. 
The services can be accessed by location, by 
client group or by service category. The kiosks 
also provide local government information, 
news, transport information and a job 
channel. 


Touch-screen kiosks in deprived and rural areas 
NHS Education Action Zones and Healthy 
Living Centres. A community pharmacy group 
partners the Dudley Education Action Zone 
(EAZ) to provide a touch-screen in the 
Netherton (a deprived inner city area) CHAT 
Centre for their “Start Here” social support 
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programme (The UK Parliament, 2003). The 
system provides some health information. 
They run a project at the local senior school 
to collect information, transfer it on to the 
touch-screen and demonstrate it to 
customers, linking in with Age Concern. This 
develops community spirit, trains a deprived 
group in IT and develops mentoring skills, 
often with younger age groups. 

Touch-screens are also available in the 
Littleport (a rural area) Healthy Living 
Centre, East Cambridgeshire. This system 
aims to improve the provision of, and access 
to, combined social and health information. 
The information collected from the “Start 
Here” touch-screens in the Dudley EAZ can 
provide a useful comparison with a similar 
exercise to be carried out in Littleport, 
highlighting the needs of deprived inner city 
versus rural areas. 

South Yorkshire Coalfields Online Project 
(SYCOP). This project is based in 
Exborough, a poor coal-mining area, with 
high unemployment and associated problems. 
A community information portal is being 
developed (SYCOP, 2003). The Web site 
covers key life events like losing a job, going to 
hospital, financial problems, etc. and aims to 
facilitate social inclusion. InTouch with 
Health is providing the health information, 
which can be accessed via the Internet at 
home, via kiosks and information points in 
public buildings and via digital TV. 

“MAGIC Touch” public information kiosks. 
Six kiosks are located in targeted deprived 
areas throughout Manchester (Dewhirst, 
2003). They were set up as part of a public- 
private-voluntary sector initiative 
(Manchester City Council, Manchester 
Community Information Network, and 
SEMA Group UK Ltd — an international 
computer software company). Information 
includes job vacancies, training courses, local 
services, council services, health contacts and 
leisure facilities, information from private 
companies and some advertising. A special 
feature of these kiosks is that anyone who logs 
on to the system has the option to create a 
personalised proforma, so that they can have 
the information they require customised to 
meet their needs, thereby saving time when 
looking for information. Use of a free e-mail 
system and printer is also available. 

Touch & Go in Glasgow. A local community 
Web site project was launched in Glasgow in 
April 2001 (Glasgow. Social Inclusion Update, 
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2001a). It aims to bring together information 
on local services using Internet kiosks to 
provide easy access for the local community. 
Touch-screens are sited in three Greater 
Easterhouse area suburbs. An offline version 
of the Web site is also available. Both formats 
incorporate a news and events section, to 
which all local groups can contribute. 
Community groups have provided the 
information for the Web site and they are also 
involved in updating it. 


Touch-screen kiosks for older people 

Older people suffering from social exclusion in 
Tameside. The Tameside Metropolitan 
Borough Council secured Single 
Regeneration Bid funding to provide better 
quality information to older people suffering 
from social exclusion (InTouch with Health, 
2003a). InTouch with Health worked with 
them to develop relevant information services 
that can be accessed via kiosks and public 
access offline computers. 

Nuneaton and Bedworth Borough Council’s 
Silver Surfers Project. This project aims to 
promote Internet usage and access with the 
over-50s. A full multi-channel solution, 
including PCs, Digital Interactive TV and 
kiosks, would be provided in January 2003 
(Neoproducts, 2003c). 


Kiosks aiming to facilitate social democracy 
Sheffield Public DataWeb. It attempts to cover 
all citizens’ digital needs and to make a 
contribution to closing the digital divide, 
whilst also being commercially deployable 
(Sheffield Public DataWeb, 2003). It targets 
the majority of people in the city who either 
do not have access to a personal computer or 
do not have the skills to use one. The 
technology was developed under a Fourth 
Framework EU Research. The concept is 
based on gathering all services and 
information together at one access point. The 
system includes popular information such as 
football scores, lottery results and horoscopes, 
which aim to encourage use of the system and 
allow users to become familiar with the 
system. They can then use their skills to take 
part in the more serious administrative and 
consultative applications, such as 
contributing to the work of the UK 
Parliament. To emphasise that the 
application requires no initial Internet skills, 
the system was launched by the oldest man in 
the city, who was 105 years old. 
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Choice Bedding Letting for Brighton and Hove 
Council 

Neoproducts produced 30 touch-screens, 
which are available at Brighton and Hove 
Council for their Choice Bedding Letting 
(Neoproducts, 2003a). People who live in 
council accommodation can access the 
screens to see what accommodation is 
available for swapping. 


Availability and use of touch-screens for 
health information and promotion 
Information about health information and 
promotion is increasingly provided using 
touch-screen technology. The first touch- 
screen health information system was 
probably Healthpoint, developed by Professor 
Ray Jones at Glasgow University in 1991 
(jones et al., 1993b). In early evaluations, 
evidence showed that people had a high recall 
of what they had read on screen ((Jones et al., 
1993a). l 

The most widely available touch-screen 
kiosks currently are: 
* NHS Direct kiosks; 
* InTouch with Health kiosks; 
* Wellpoint Health Centres; and 
°. The Patient Information for Consent 

Systems. 


NHS Direct Kiosks 

A total of 186 NHS Direct kiosks are 
currently available in various locations in the 
UK. This service forms part of the 
Government’s NHS Plan, aiming to provide a 
convenient out-of-hours health service 
(Action Multimedia, 2003). The kiosks 
facilitate e-government, together with other 
new technology projects currently piloted, i.e. 
information through digital interactive TV 
and 3G mobiles; they also assist in managing 
staff capacity in NHS Direct Helplines. The 
NHS Direct information points have twin 
aims: to give the public greater information 
about how they can look after their own 
health and also to give information about 
health services. 

The health information on the kiosks is 
similar to that found on the online NHS 
Direct Web site. The kiosks are situated in 
key public locations such as pharmacies, 
supermarkets, holiday resorts, libraries, 
shopping malls, universities, sixth form 
colleges and more traditional NHS locations 
such as AXE departments and NHS Walk-in 
Centres. The Department of Health 
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commissioned independent consultants to 

help develop the specifications for the kiosks, 

including disability and access options. 

Action Multimedia has provided kiosk 

hardware, consultancy, pre-installation site 

surveys, remote monitoring and software 
updates. The first NHS kiosk was installed in 

September 2000 and the project was 

launched in October 2000 (Action 

Multimedia, 2003). A total of 81 kiosks were 

installed by February 2001, 136 by October 

2001, and 180 by January 2003. A number of 

the NHS Direct Information Points feature 

telephone handsets, enabling the public to 
contact their local NHS Direct Call Centre 
free of charge, for additional support and 
health-care advice. 

Evaluations and usage statistics. Initial 
evaluation of usage of 136 NHS kiosks up to 
October 2001 (NHS Direct, 2001) showed 
that: 
¢ Kiosks were used on average 12 times per 

day, but many kiosks could have been 
better sited; a third of those visiting sites 
had noticed the kiosk and 6 per cent had 
used it. 

¢ A quarter had obtained health 
information from the Internet, 36 per 
cent had not used the Internet for health 
information nor would they use the kiosk, 
but 39 per cent would use the kiosk now 
they knew it was there. 

+ Sites such as supermarkets and leisure 
centres which are visited frequently by 
people aged over 60 would be most likely 
to reach those who do not use the 
Internet for health information. 

e Although a greater proportion of those 
visiting health centres and community 
centres used the kiosk, the greatest use 
was in NHS Walk-in Centres, leisure 
centres and supermarkets because of a 
much greater number of visitors at those 
sites. 


In conclusion, it was found that, although 
more work is needed on where to site kiosks in 


_order to make sure people are aware of them, 


the kiosks can increase accessibility of health 
information. 

Each kiosk automatically monitors its own 
usage. Reports are produced regularly for 
each kiosk by Action Multimedia (Hardy, 
2003a, b, c). Most recent statistics show that 
the kiosks had 920 users per month on 
average (per kiosk) in spring and summer 
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2002, but there are many variations in usage 

by location and area. 

Limitations. Although these kiosks can 
increase accessibility, better siting could 
improve awareness (NHS Direct, 2001). 
According to the consultations, there are 
some other limitations: 

e Social inclusion. Although the NHS Direct 
Helpline uses audio-tapes with 
information in ethnic languages and NHS 
Direct Online has some non-English 
content, information on the kiosks is 
currently only available in English. 

¢ Location. The decision on where to locate 
kiosks is currently taken by the local NHS 
Direct Centres. This results in lack of 
comparability between different areas. 
Location is very fragmented, and local 
users do not have a choice in where kiosks 
are located. 

e Design. The current kiosks are quite old- 
fashioned and bulky; this presents some 
problems in some locations when space is 
limited. 

* Interactivity. They are not live or Web- 
connected; the software is loaded in a 
CD-ROM, thus not media-interactive. 

* Local information. Local information, i.e. 
about hospitals or services in the locality, 
is not currently available. 


Future. There is an NHS Plan commitment 
for provision of NHS Direct information 
through 500 kiosks by 2004. According to the 
consultations, the information may be 
available in NHS Direct kiosks and/or other 
kiosks by either increasing the number of 
NHS Direct kiosks or having licensing 
arrangements with third parties to provide 
NHS Direct information using the real estate 
available on other existing kiosks. The second 
option may result in losing some of the NHS 
branding image, but would increase 
accessibility — one of the objectives of the 
NHS Plan. 

There are discussions currently on how to 
overcome the other limitations mentioned 
above. NHS Direct aims to identify ways of . 
making information available to black and 
minority ethnic groups (NHS Direct, 2002) 
using new technologies such as digital 
interactive TV, 3G mobiles and hand-held 
devices. Short-term priorities include access 
to computers in local communities and access 
to training needs for computer usage to be 
publicised. Long-term priorities include 
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building Web access for disadvantaged 
groups. The possibility of having touch- 
screen kiosks with NHS Direct information in 
local communities, with content in local 
ethnic languages, is also discussed. 


InTouch with Health kiosks 

About 170 InTouch with Health kiosks are 

available, 45 per cent of which are located ina 

primary care setting, 45 per cent in hospitals 
and acute trusts and the rest in a mixture of 
sites such as libraries, community sites and 

pharmacies (InTouch with Health, 2002). 

They combine a simple “touch button” 

method for viewing information and printing 

out factsheets and a selection of Web sites, 
which can be chosen by the client and 
accessed using the kiosk’s in-built keyboard 
and roller ball. The newest versions of kiosks 
are Web-enabled. Some InTouch with Health 
systems are listed as follows: 

« Patient Advice and Liaison Service in 
Bradford has been organised on a 
metropolitan basis (InTouch with Health, 
2003b). Kiosks have been installed in 
three hospitals; the scheme will be 
extended into primary care in 2003. They 
provide comprehensive national and local 
information as well as allowing patients to 
give feedback to the central PALS team. 
Information is available in some minority 
languages. 

¢ Patient Advice and Liaison Service in 
Whipps Cross University Hospital NHS 
Trust has also installed kiosks as part of a 
wide-reaching communication plan that 
incorporates PALS information with 
interactive material explaining the 
treatment carried out in the specialty 
areas associated with the hospital. 

¢ West Pennine Health Authority has 
installed a network of kiosks across 
primary and secondary care as part of a 
Local Implementation Strategy project to 
involve patients in their treatment and 
help them to manage their conditions 
better (InTouch with Health, 2003a). 

* Harrogate General Hospital has recently 
opened an information centre containing 
a Web kiosk and sit-down booths for 
extended use. Special developments 
include access to patient videos via the 
kiosks. 

* Oldham NHS Trust has installed a 
number of kiosks throughout the hospital 
in order to provide public access to 
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quality health information, which also 
includes sections in Gujarati, Bengali, 
Urdu and Chinese (InTouch with 
Health, 2003c). 

+ Edinburgh Royal Infirmary has placed two 
kiosks in the information centre and the 
A&E department. 

* Hadfield Medical Practice has installed a 
kiosk, which is manned by members of 
the Patient Partnership Group (InTouch 
with Health, 2003d). It is located in the 
“Information Room” in the Medical 
Centre, an area of the waiting-room 
devoted to health education and 
information. A printing facility is 
available. 

e Phoenix Surgery has set up an information 
area within the practice with an InTouch 
kiosk as its centrepiece. 


Wellpoint Health Centres 

The Wellpoint Interactive Health Centres 

(Neoproducts, 2003d; Greener, 2002) 

feature: 

* Health tests option. This measures blood 
pressure, heart rate, weight, body mass 
index and body fat composition. Other 
tests are available and will be introduced, 
as the machines become more familiar to 
users. 

* Health information option. This features a 
searchable database of over 500 common 
ailments for health information, provided 
by the College of Health. A database of 
2,500 health-related self-help groups will 
be also introduced. 


The users can store their personal history, up 
to ten results on the system, accessed via a 
PIN code. The kiosks also include a printer 
facility, coin mechanism, stereo speakers, 
guardian kiosk management module and 
lighting display “wings”. The units have been 
built with upgrade-ability in mind and a new 
set of health-monitoring options is already in 
the pipeline. They currently operate with 
software installed locally (CD-ROM), but 
Web-versions will be available in the future. 

The system dovetails with the 
Government’s desire to create empowered 
patients. Used as a highly advanced 
monitoring tool, the system could offer 
pharmacists the chance to improve patients’ 
health as well as advancing the profession 
more generally. As the company’s managing 
director said: 
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The system is unique and represents a major 
opportunity for pharmacists to firmly establish 
themselves as the first port of call for health 
monitoring and advice. 


The Wellpoint Centre is currently available in 
30 UK independent pharmacy outlets and is 
being rolled out to an initial total of 200 
independent pharmacy outlets. It is targeted 
to be available in over 2,500 locations by the 
end of 2004. 


Patient Information for Consent System 
Touch-screens are used as a secure delivery 
platform for a Patient Information for 
Consent System (Neoproducts, 2003e; 
Patient Info, 2003). The main purpose of the 
system is to offer patients information on 
medical procedures in a clear and concise 
way, to educate and empower and to facilitate 
consent through education. They provide 
benefits not only for patients, but also for 
clinicians, as ‘the system can enable more 
effective and efficient doctor/patient 
discussion due to pre-consultation patient 
education, improve the doctor/patient 
relationship and reduce the likelihood of 
complaints and litigation due to a full and 
proper consent process. 

The software is installed on touch-screen 
kiosks, which present the information and 
also print out patient information leaflets. It 
uses text, voice-over and photographs/ 
diagrams to describe the particular procedure 
that a patient is to undergo. It includes 
associated information about risks, 
complications and alternative treatments. It 
allows the patient or doctor to print patient 
information sheets and government pre- 
written consent forms, which can be 
customised for the host hospital. The kiosk 
has full disabled access, especially for patients 
in a wheelchair and for those seated at the 
unit. There are also stereo speakers and a pair 
of stereo headphones. The Patient Consent 
Form is printed on a grade of paper designed 
to last for up to 25 years. 

The systems are currently being rolled out 
in hospitals and private clinics, as either 
Internet or intranet versions. Approximately 
ten systems are currently available in London 
(Hackney PCT, Whipps Cross University 
Hospital, and the London Spine Clinic), 
Essex (intranet version) and Wolverhampton. 
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Other touch-screen kiosks 

Other touch-screen kiosks are available on a 

much smaller scale or for people with specific 

conditions: 

* City-Wide Health Information Service in 
Glasgow. This project has been jointly 
developed by the Mitchell Library and 
the Primary Care Trust’s Sandyford 
Initiative (Glasgow Social Inclusion Update, 
2001b). A wide range of health topics is 
available on the Web site with links to 
other health Web sites through touch- 
screens. It is expected to expand the 
number of touch points and facilities to 
be made available in all libraries in 
Glasgow and more Primary Care Trust 
sites. 

+ Scream Ltd kiosks. These kiosks mainly 
provide local information about the trusts 
or the councils (Scream, 2003). They are 
available to users of 12 Trusts and 
councils around the UK. 

*  Touch-screens with rheumatology 
information. St Thomas’s Hospital in 
London, Rheumatology Department, 
have created a simple touch-screen for 
rheumatology-related topics (Technology 
Active Solutions, 2003a). 

* Touch-screens with cancer information. 
They are available in the Cancer Patient 
Information Centre of Wales 
(Technology Active Solutions, 2003a). 


Availability and use of touch-screens 

with health information and health 

promotion content targeting socially 

excluded groups 

We have identified some projects, which 

specifically target socially excluded groups 

and other population sub-groups: 

* touch-screen kiosks with information in 
ethnic languages; 

* touch-screen kiosks for older people; and 

* touch-screen for other (socially excluded) 
sub-groups. 


Touch-screen kiosks with information in other 
languages 

Most of the systems detailed can be voiced or 
translated into various languages, thus 
combating the problem of availability to 
ethnic minorities whose first language is not 
English (Patient Info, 2003). One of the big 
problems with people from ethnic minorities, 
especially older people, is that, even if they 
can speak the language, they probably cannot 
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read it or write it. Therefore it is beneficial to 
translate in written form and also have spoken 
word translations. However, this can be 
expensive. 

Touch-screens with health-related 
information in ethnic languages have been 
produced by InTouch with Health. 
Information in some ethnic minority 
languages is available on PALS Bradford 
kiosks; information in Gujarati, Bengali, 
Urdu and Chinese is available on Oldham 
NHS Trust kiosks. However, the “Three 
cities” project has made health-care 
information widely available to ethnic 
minorities. 

The “Three Cities” project. A partnership 
undertaken by health specialists InTouch 
with Health and Sheffield, Nottingham and 
Leicester Health Action Zones and initially 
funded for three years InTouch with Health, 
2003c) has led to minority groups gaining 
access to more health- and social-care 
information and advice in their own 
languages. As the project manager said: 

We had to find a way to break down the 

communication barrier faced by ethnic 

minorities and there were a number of problems 
to overcome. Some languages only have a verbal 
tradition; so many people are unable to read 
their mother tongue. We tried to develop 
something that will enable people who are not 


necessarily literate in their own language to 
access the information, which is a real challenge 


Four languages were decided upon: Chinese, 
Bengali, Gujarati, and Urdu/Punjabi in 
addition to English. Each Health Authority 
led the development of their chosen language. 
Local steering groups, including community 
representatives and health workers, were then 
set up to look at the specific health needs of 
each City and prioritise the information 
subjects to be available. Research revealed a 
number of conditions prevalent amongst the 
ethnic groups in all three areas: heart disease 
prevention, diabetes, tuberculosis and cervical 
screening. 

The kiosks include a handset to enable 
users to listen to the health messages and clear 
touch-screen instructions for written 
information. Information can be printed off 
and taken home. To encourage people to use 
the kiosk, a continuous video clip, which 
shows someone from each community 
walking up to the kiosk, touching the screen 
and picking up the handset, was built. Users 
can select the language of their choice and 
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then choose the topic from a simple menu. 
There is also a demographic form for people 
to fill before they use the information further. 

The sites for the touch-screen health kiosks 
have been carefully selected and are changed 
during the project. Locations include 
libraries, health and medical centres, 
neighbourhood centres, ethnic minority 
centres, and a mosque. 

The touch-screens are being evaluated by 
the University of Sheffield. The system is 
monitored online and a questionnaire study 
was conducted over 18 months (July 2001- 
December 2002). Current feedback indicates 
great support from people who can hear and 
see basic health information in their own 
language for the first time; interestingly, many 
users share the information with family and 
friends. The kiosks were particularly 
successful with the Hindu community when 
they were located in a Hindu temple. 

Limitations. There are only three points of 
interaction, one in each city; the continuous 
rotation of the screens does not allow for 
continuity of use and shared ownership of the 
project with users. Continuation and 


ae _egpansion of the project depend on funding. 


Touch-screen kiosks for older people 

It has been shown that computer and Internet 
usage is dramatically affected by age 
(Coleman et al., 2002). Senior citizens are 
considered not as technically adept as their 


.. younger counterparts and are often frightened 


by computers, especially keyboards 
(Mendelsohn, 2001). Evaluation data, 
however, on touch-screen use in public 
settings show that older people use them; 15 
per cent of users at Creswell Crags Visitor 
Centre, where touch-screens were placed, 
were over 60 years old in 1995 (Technology 
Active Solutions, 1995). However, only one 
project with health prevention and 
health-care information targeting older people 
was identified and this is still under 
development. 

Falls program for older people in Tower 
Hamlets communities. Implemented as a Web 
site and a touch-screen kiosk (supplied by 
InTouch with Health) application, the Falls 
program aims to provide health information 
to older residents, their carers and families in 
the London Borough of Tower Hamlets, who 
are at risk of a fall (Wilmes et al., 2003). By 
providing facilities to screen for and flag risks 
of falling, the final application will also be able 
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to channel vulnerable individuals into the 
health-care system. It is expected that the 
application will increase awareness of the 
consequences of falling, in both those at risk 
and those in primary care. Initially three 
access points will be available: one in the 
hospital, one in a local library, and another 
one in a GP practice. There is scope for other 
areas to be developed at a later stage, such as 
information about stroke and medications for 
older people. 

Focus groups and in-depth interviews are 
organised to investigate older people’s attitudes 
towards health information on falls and their 
experiences with computers/the Internet and 
accessing online health information. The 
diversity of the Tower Hamlets population is 
also taken into account. Throughout the entire 
process of design and implementation a series 
of usability tests (questionnaires and 
observational research) will be conducted to 
evaluate the program’s usability and 
accessibility, i.e. intuitiveness, design for users 
with visual and cognitive impairment, design for 
users with reduced motor coordination skills, 
presentation of content for people who cannot 
read, provision of content in languages 
prevalent in the community, usage of different 
multimedia elements (to cater for different 
learning styles). 


Touch-screen kiosks for other (socially excluded) 

sub-groups 

InTouch with Health provides other kiosks 

targeting specific subgroups of the population 

(InTouch with Health, 2003a): 

* Hindley Prison kiosks. Kiosk technology 
was employed to provide reliable 
information to both prisoners and 
visitors. Special drugs information was 
added on to the kiosk at the prison’s 
request. 

« South Essex Mental Health Partnership 
Trust. A total of 45 kiosks were supplied 
with special mental health content. More 
ethnic content to reach the big Polish 
community in the area they serve will also 
be created. 


Elements of successful usage and 
factors which determine usage 


A number of important elements that may 
contribute to successful usage of touch-screen 
technology were identified: 
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+ A simple interface that takes into account 
the computer skills and the educational 
level of possible users (Cranston et al., 
1996). 

e Screen design, colours, the use of levels 
within a hierarchy and easy navigation 
(Cranston et al., 1996; Technology 
Active Solutions, 2003b). 

* Internet access, or at the very least 
intranet access. 

e An interactive multimedia program and 
software to provide an automatic return 
to loop or screen saver when the kiosk has 
not been touched for a certain length of 
time. 

* A built-in survey tool to collect feedback 
from users (Cranston et al., 1996). 

* Monitoring tool, log file generation to 
track users, and statistics generation for 
management and potential advertisers. 

* Accessibility and ease of use for everyone 
are equally important — “Keep it Simple 
(&) Stupid” (Mendelsohn, 2001). 

* Requirements of groups with little or no 
computer literacy and those with special 
needs, visual impairments or other 
disabilities (Ekberg, 1998; JSC, 2003). 


Most of the companies have their own quality 
criteria InTouch with Health, 2002) ora 
user group (Technology Active Solutions, 
2003a) to advise them on design and 
development, as they realise that users’ needs, 
experiences and confidence are not uniform. 
As one user said (Technology Active 
Solutions, 2003b): 


You can’t just stick a terminal in a public place 
and expect people to use it. They won’t. Nor can 
you expect a 60-year-old with arthritis to “rotate 
the ball until the pointer is over the menu you 

require, scroll down to the action you need and 
select it with a rapid click of the primary button”. 


At least one of the companies has put together 
approval criteria for disabled access such as 
access to a touch-screen. by lying on their 
sides, sufficient width and depth for the 
wheelchair, appropriate height and within 
easy reach screen, size of the screen, type of 
touch technology and physical stability 
(Technology Active Solutions, 2003b). Other 
companies also consider disability and access 
issues, and their screens are tested rigorously 
to meet specification criteria (Neoproducts, 
2003a). Access and use by children and 
elderly people have also been considered. 
Elderly people may have specific needs; as 
one of the developers said: 
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Somewhere to put their walking-stick on, as they 
commonly like to lean on the touch-screen, as it 
is more steady than their walking-stick. 


Other factors that may influence the 
accessibility of touch-screen information for 
patients were outlined in evaluations of kiosks 
(Pearson et al., 1999; Nicholas et al, 2001): 

¢ faith in the content (i.e. the question of 
authenticity); 

* ease of use — hardware, navigation and 
other aspects of the interface are 
important; 

* convenient location at a convenient time; 
and 

* cost — plainly an attraction of kiosks is 
that they are free to use. 


Consultations with stakeholders highlighted, 

in addition, the importance of the following: 

« the quality and the usefulness of the 
content; 

* the degree of ownership or responsibility 
the organisation has for the kiosk and its 
promotion; 

* access to information to which 
authorities/organisations want people to 
have access; and 

* access to information (information, data 
about themselves) to which people would 
like to have access. 


The future of touch-screens 


There was an agreement between the 
stakeholders that community- and health- 
related information, which should be readily 
available to people, would be provided widely 
in the near future via new technology. 

Other points made were: 

* More touch-screens with a keyboard or 
trackball will be more widely used, as the 
computer-literate younger generation 
grow older. These will be accessed via ID 
cards or Smart cards, or free. 

* TVs with Web browsers are also another 
possibility. The view was expressed that, 
although digital TV (free to view 
channels) is a good idea, users still need 
to be equipped with the tools — which are, 
however, relatively low-cost — to access 
the Web. 

* Biometrics, eye scanners and print 
scanners are considered to be more 
advanced technologically and they will be 
more widely available in the long term. 
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The majority of stakeholders expressed the 
view that kiosks should be available in a 
variety of locations irrespective of their 
specific content. Only one stakeholder felt 
that kiosks that provide information about 
health should be located in sites which people 
already associate with health, for example, 
surgeries, pharmacies, and health-related 
areas of supermarkets. 


Conclusions 


This study shows that kiosks in general are 
starting to play a major part in allowing public 
access to information. Touch-screen kiosks 
have been used to provide information and 
choice in employment, community services, 
local services and health. They have been 
used to provide information and offer choice. 

New technology and touch-screens in 
particular seem to be appropriate vehicles for 
provision of information in various settings, as 
they provide a number of benefits: 

* kiosks can be used by people who do not 
have access to a computer at home or 
work; 

« kiosks can provide quick access and 
access to information 24 hours per day; 

* they are easy to use; 

* users with little or no computer literacy 
do not feel intimidated by the technology 
(Cranston et al., 1996); 

*  touch-screens are non-threatening and do 
not frighten older users away 
(Mendelsohn, 2001); and 

+ they also facilitate a sense of ownership 
and choice. 


Touch-screens facilitate social inclusion, even 
if they do not specifically target specific 
groups. They are addressing a socially 
excluded group, those who have no access to 
or the skills to use computers. Siting kiosks in 
public is helping to bridge the gap between 
the “IT haves” and the “IT have-nots”. 

In addition, information kiosk technology 
can provide the means through which it is 
possible to access socially excluded groups in 
a non-threatening and informative way. The 
kiosks can be placed in situations where 
people can access information who may 
otherwise miss out. They offer a degree of 
anonymity, which is not possible through 
other methods. There is also the potential to 
be used widely by specific subgroups of the 
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society. This technology provides us with an 
opportunity to provide social and health 
benefits (Nicholas et al., 2000). 

Care should be taken to ensure that some 
parts of society are not disadvantaged as a 
result of wrong assumptions about the 
ubiquity of the Internet. Computer and 
Internet usage is dramatically affected by age. 
The proportion of those above the state 
pension age with any experience of the 
Internet is significantly lower than average 
(Coleman et al., 2002). Computer and 
Internet usage is also dramatically affected by 
command of the English language. To a lesser 
extent, the level of Internet usage is also 
affected by socio-economic factors such as 
work status, income levels and qualifications 
obtained. However, these may change in the 
future, as more and more people, irrespective 
of the factors mentioned, are aware of 
electronic service delivery and new 
technology. A key part of ensuring take-up 
will be to ensure high levels of awareness of 
these services among those groups likely to 
want to use them (Coleman et al., 2002). 

One of the most difficult challenges in 
public health is how to get the message across 
to socially excluded people in a form that will 
be meaningful to them. It seems that touch- 
screen technology, with kiosks located in 
easily accessible public locations, may 
overcome this problem. However, further 
research on touch-screen technology and its 
usability with regard to various subgroups of 
the population, including socially excluded 
groups, is needed. 
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Abstract 
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Introduction 


Equal access to health information for 
patients and their families is a global issue but 
also important between different groups 
within richer countries such as the USA and 
the UK (Mand! et al., 1998; Servaes and 
Heinderyckx, 2002; Brodie et al. 2000; 
Hughes er al., 2002). The idea of accessibility 
to health information has two components: 
physical availability and personal 
retrievability. 


Physical availability 

Physical availability can be through publicly 
available, socially available or privately 
available sources of information. Examples of 
publicly available sources of information are a 
cancer caring centre, public library providing 
Internet access, or a newspaper or magazine 
widely distributed. Although a source is 
publicly available, it may still have a cost (for 
example, to buy or to travel to use). A socially 
available source is information available 
through someone else, via a significant other, 
family members, neighbour, or colleague who 
may have access to the Internet. Finally, 
privately available information is through, for 
example, TV at home, Internet at home or 
work, telephone advice line at home. 


Personal retrievability 

Personal retrievability is whether or not 

the person has the money, time, skill, 
knowledge, mobility, understanding, and 
emotional strength to make use of a particular 
source of information. This depends on the 
complexity of different information sources or 
the relevance of the information presented. 


Social support 

Social support may affect both physical 
availability and personal retrievability. 
Instrumental and informational support can 
affect physical availability, i.e. someone can 
offer the use of their computer or can drive 
someone to a library to let them use one there. 
Informational and emotional support can 
affect personal retrievability by helping 
someone to understand the information they 
have or by emotionally supporting them to ask 
questions or go to find out about something. 


Information use 
Information use (i.e. the number and type of 
sources of information that a patient or family 
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members will access) will depend on their 
perceived need for information (patients who 
do not want information will not seek it) and 
perceived accessibility. 


Satisfaction with information 

Satisfaction with information received will 
depend on whether there was sufficient use to 
meet the need, modified by the perceived 
appropriateness of the information obtained, 
but also by their expectations and personality. 
For example, we found that breast cancer 
patients expected more information than 
prostate cancer patients and that patients who 
scored highly on a depression scale were more 
likely to be dissatisfied with the health 
information they had received (Jones et al., 
1999). 

This paper illustrates some different ways in 
which accessibility can be improved. These 
range from improving physical availability 
through touch-screen kiosks to tailoring - 
information to the individual, thus making it 
more relevant, less overwhelming and more 
likely to engage the social support of other 
family members. 


Touch-screen kiosks 


One approach to tackling the physical 
availability of computer-based information is 
to locate touch-screen kiosks or booths in 
community settings. Healthpoint was a 
community-based touch-screen health 
information kiosk for the public. The first 
Healthpoint was developed and evaluated in a 
health centre in Glasgow in 1989 (Jones and 
McLachlan, 1990). A subsequent study in 
1992-1993 used a variety of evaluation 
methods including: automatic monitoring by 
the computer, interviews with users and non- 
users, postal questionnaires, a random 
telephone survey, and observation of users 
(ones et al., 1993a, b). Healthpoint kiosks 
were sited in supermarkets, shopping centres, 
community pharmacies, health centres, 
hospitals, bars, sports centres, post offices, job 
centres, and libraries amongst the 23 sites 
tried (Plate 1). There were few problems in 
finding sites for the kiosks. A random 
telephone survey of 231 people in 1992 found 
that 17 per cent of respondents had used one 
or more of ten kiosks sited in Clydebank for a 
five-month period. The prevalence of users: 
amongst the over-50s (13 per cent) was not 
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Plate 1 Healthpoint in use in a Glasgow shopping 





much less than the 20 per cent of users aged 
under 50 (whereas the difference in level of 
current use of the Internet by age is much 
greater). Interview data showed a more 
positive reaction from the “less educated” 
than the educated and was expected because 
of the style of presentation. In a subsequent 
study in one sports centre in 1996, we found 
that 9 per cent of people who had used the 
kiosk had apparently had no other form of 
health information in the previous two weeks 
Naven et al., 1996). 

Other touch-screen health kiosks have been 
developed, including a commercially available 
system called InTouch with Health[1] and an 
Anglo-Spanish system focusing on 
medications and aimed at community 
pharmacies[2]. To improve access to both 
health information and health services in the 
UK, the UK Government established NHS 
Direct in 1998. The main channel of 
communication has been a single point of 
entry, telephone helpline operating through 
22 NHS Direct call centres. Last year NHS 
Direct received around 12 million calls and 
this is expected to rise to 18 million in the 
current year. In addition, the NHS Direct 
Online Web site[3] offers a wide range of 
health information including access to local 
contacts and links to other sites. The Web site 
currently has around 250,000 visitors a 
month. To improve equality of access, NHS 
Direct installed touch-screen kiosks between 
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September 2000 and.October 2001 in 136 
locations, including NHS “walk-in centres”, 
community centres, retail pharmacists, 
hospitals, public libraries and retail sites. The 
information provided was similar to that on 
the NHS Direct Web site. 

A study in the summer and autumn of 2001 
assessed whether NHS Direct touch-screen 
kiosks can increase access to health 
information (Jones, 2001). Routine usage 
data were collected from 136 NHS Direct 
kiosks and postal and interview surveys were 
carried out for a purposive subsample of 20 
kiosks. Kiosks had been sited in 
supermarkets, libraries, retail pharmacies, 
leisure centres, community centres as well as 
NHS sites. Kiosks were used on average 12 
times per day but many kiosks could have 
been better sited. A third of those visiting sites 
had noticed the kiosk and 6 per cent had used 
it. Two-thirds would use it in the future. Four 
out of five aged under 30 had access to the 
Internet compared with only 24 per cent over 
60. A quarter had obtained health 
information from the Internet, 36 per cent 
had not used the Internet for health 
information nor would they use the kiosk, but 
39 per cent who had not used the Internet 
would use a kiosk now that they knew it was 
there. Sites such as supermarkets and leisure 
centres which are visited frequently by people 
aged over 60 would be most likely to reach 
those who do not use the Internet for health 
information. Other studies have shown the 
importance of siting kiosks appropriately 
(Jones et al, 1993b; Nicholas et al., 2002) and 
more work was needed on finding good sites 
for these NHS Direct kiosks and making sure 
that people are aware of them. However, the 
study provided some evidence that kiosks can 
increase accessibility of health information. 


Access to computer-based information 
for particular patient groups 


There are many patient groups where 
information and education are part of the 
treatment. For example, we developed a 
multimedia system for cognitive behavioural 
therapy in the management of anxiety (White 


et al., 2000). In a pilot study 29 patients had 


booked appointments to meet a research’ 
assistant and use a computer in a health 
centre. Finding space to site the computer . 
was difficult and it was kept on a trolley and 
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‘taken to an empty room by the research 


assistant for each appointment. Most of the 
patients showed good improvements in 
anxiety over six months. However, we needed 
to demonstrate its effectiveness in routine 
practice in primary care. 

To address the problem of location for the 
computer we explored sites for the computers 
in both health centres and public libraries in 
Glasgow. We were able to site only one 
computer in a health centre but were able to 
site computers in most public libraries (Plates 
2 and 3). In a randomised trial, GPs referred 
patients to be recruited by the project at ten 
public libraries and one health centre. Of 239 
patients referred by their GP, 178 (74 per 
cent) were recruited to the study, the majority 
attending at a public library. This 
demonstrated that public libraries could be 
used as a referral destination for patients 
requiring information-based “treatment”. 
Virtually all the UK’s public libraries now 
offer Internet access and a trial of their use in 
urban centres would be worthwhile. 


Plate 2 Multimedia touchscreen booth being used in a 
Glasgow public library 
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Plate 3 Multimedia touchscreen booth being used in a 
Glasgow public library 
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The difficulty of relying on just one location 
for a computer system was illustrated in a 
randomised trial of personalised computer- 
based education vs. CPN education for 
patients with schizophrenia (Jones et al., 
2001). In this study a touch-screen computer 
was sited in a Day Centre in the Gorbals 
Glasgow for use by patients with 
schizophrenia. Patients recruited to the study 
were allocated to use the computer for five 
sessions, to see a community psychiatric nurse 
(CPN) for five sessions, or to have a 
combination of one session with the CPN, 
three with the computer, then another with 
the CPN. The main aim of the educational 
package was to improve knowledge about 
schizophrenia without worsening mental state 
or insight. We expected the computer option 
to be cheaper than the CPN. No major 
differences were found in knowledge, insight, 
or mental state over three months between 
computer and CPN. One third of those in the 
“combination” group preferred the computer. 
However, the computer was no cheaper, as 
patients had to be collected and taken to use 
the day centre-based computer by the 
research assistant to ensure attendance. We 
modelled the costs of other options. A 
cheaper option would be to spend more 
capital on providing a number of out-patient 
clinics with suitable computers, so that use of 
the computer could be combined with other 
visits, such as for a monthly “depot” injection. 


Tailored information 

As the volume of health information increases, 
accessibility starts to be compromised, as 
patients become overwhelmed with irrelevant 
information. We tested the hypothesis that 
information tailored to the individual will be 
“better” than more general information (Jones 


Figure 1 Examples of screens in “personalised systems” 


The Record of Donald Demo 


The information in your record has been divided- 
into the following 4 categories. Please touch one of 
the highlighted words to see your details. 


There is also general information on the following 
topics: 
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et al., 1999). A total of 525 patients attending a 
Glasgow hospital for radiotherapy were 
recruited. They were randomised either to use a 
computer offering information based on their 
medical record, or general information about 
cancer (such as they would see on a quality 
Web site), or to be offered a full range of 
booklets. (Figure 1 shows examples of “medical 
record” screens.) We provided both interactive 
access to information (at a special booth at the 
hospital) and computer print-outs of the 
information explored. We measured patient 
satisfaction, use and reuse of the computer, and 
cost. Patients who had tailored information 
were more satisfied with the information and 
more likely to show the printed copy to their 
family. 


Social support 


Some previous authors had claimed that 
giving cancer patients access to their own 
medical records might increase anxiety, so we 
measured it over the three months of the 
study. We expected no difference but 
surprisingly found that patients with general 
information showed less improvement in 
anxiety than those with information based on 
the medical record. We hypothesised that this 
may be due to patients with personalised 
information being more likely to discuss the 
information at home. We have just completed 
the data collection in a randomised trial to 
explore that hypothesis, focusing mainly on 
the paper format[4]. (A paper leaflet has the 
advantage that it is a resource that may be 
shown and discussed with family and friends. 
Internet access is often isolating.) 

We have recruited 400 patients and their 
“significant others” to the study. Data 








Personal Information 
Investigations 
Problems 
Treatments 


About Cancer 
Beatson Cncology Centre 


PACKE 
START 


Donald Smith: Prostate Cancer 


According to your record, you are being treated for 
this problem. 


Your prostate cancer is described in medical words 
as being a grade 2+3, localised, adenocarcinoma. 


The cancer was sited in the left lobe of your 
prostate gland and had affected the capsule 
surrounding it. One of your seminal vesicles may 
also be affected by the cancer. 


Your cancer was staged according to the TNM 
System as T3, 
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collection is finished but we have yet to 

complete the analysis. Our focus is on how 

different approaches to leaflet provision 

impacts first on the social interactions 

encouraged, and second on levels of anxiety. 

First, we will model use of booklets in the 

home as a function of whether: 

* patients chose the information themselves 
or if it was chosen for them; 

* it included personal information from 
their own medical record or not; and 

* it included advice on the management of 
anxiety. 


Second, we will model anxiety at three 
months as a function of these three factors 
and patients’ use of leaflets in the home. Our 
hypothesis is that personalised information is 
more likely to be shared with the “significant 
other” and that sharing information will help 
reduce any anxiety experienced for both. 


Conclusions 


Many NHS and other policy documents state 
the need to make health information accessible 
to all and there have been a number of 
initiatives (both by NHS and in the context of 
funded research) to achieve this. However, the 
concept of accessibility has rarely been defined. 
Being able to access health information via the 
Internet at home is not the only measure of 
accessibility. We may be able to access 
computers via kiosks, at the public library or in 
health centres. However, physical access is not 
sufficient and more thought is needed as to the 
format and content of the information and the 
social context in which it is used. In all cases we 
need more evidence from well designed studies 
to inform the debate. 


Notes 


www. intouchwithhealth.co.uk 
http://dis.sema.es/projects/TESEMED 
www.nhsdirect.nhs.uk 
www.cee.hw.ac.uk/~diana/paccit/index.htm 
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This article is a summary of a report which looks at the 


- potential of using television sets at hospital bedsides to 


provide patients with access to health information. The 
report was jointly commissioned by The Department of 
Health (DoH) and National Health Service (NHS) Direct 
Online, and focused on what could be done using existing 
(or planned) Patient Power infrastructure. The latest 
generation of systems from all three of the main suppliers 
of bedside television service could provide access to 
health information in a variety of ways. A group meeting 
was held with key internal DoH and NHS stakeholders. 
Interviews were held with the main suppliers of bedside 
television services. Opinions were also taken from several 
academics specialising in Health Informatics, from a range 
of commercial health information providers and a wide 
range of NHS Trust representatives. Two areas where 
further research would be of use are looking at the 
opportunities around digital radio as a way of providing 
patients with health information, and looking at any 
particular issues that exist around communication 

with patients while they are in hospital. 
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introduction 


Bedside TV and telephone systems are 
currently being installed in hospital Trusts 
throughout England under a programme 
titled Patient Power, managed by NHS 
Estates. These services are currently being 
deployed only in hospitals with 150 or more 
beds on one site, representing 255 hospitals, 
totalling approximately 130,000 beds. There 
are two categories of organisation that have 


-been issued with Patient Power licences by 


NHS Estates: full licence holders and 
provisional licence holders. 

Full licence holders have demonstrated 
their ability to produce and maintain a patient 
television and telephone service. There are 
four organisations that hold full licences, and 
all are competing with one another to secure 
contracts with Trusts to provide their bedside 
television and telephone services. The 
suppliers have to bear the cost of installing 
and maintaining the systems, and can then 
generate revenue (and in time, they hope, 
profit) from patients paying to watch 
television and make telephone calls. There is 
no cost to the NHS for the provision of the 
basic entertainment services. 

The systems being installed are there 
primarily to provide a bedside entertainment 
system for patients, with direct access to 
television and a telephone. The same 
infrastructure could be used for a wide range. 
of additional purposes, such as providing 
electronic access to patient records, or access 
to clinical decision support software, as well 
as having the potential for providing patients 
with access to high quality health information, 
which was the focus of this report. 

Patientline has been operating television 
and telephone services longer than any of the 
other suppliers. Hospital Telephone Services 
(HTS) and Wandsworth Electrical also have a 
number of sites that are now operational. The 
fourth full licence holder, Kerfone, is 
operational in one hospital. 

Provisional licence holders need to 
demonstrate to NHS Estates their ability to 
produce and maintain a Patient Power service 
in one Trust before they can extend that 
service to other hospitals. Three of the 
provisional licence holders are working with 
Trusts to demonstrate that capability 
(Sodexho, Premier Managed Payphones, and 
Patientpal). 
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There are a number of areas where 
televisions and telephones at the bedside may 
not be installed. It does vary in different 
Trusts and on the recommendations made by 
suppliers (such as beds dedicated to dialysis, 
treatment of the very elderly, intensive care, 
and areas for the treatment of mental health 
patients). 

It is thought that sites with fewer than 150 
beds are unlikely to be economically viable 
under the terms of the Model Concession 
Agreement prepared by NHS Estates (a copy 
of the Model Concession Agreement is 
available at: www.nhsestates.gov.uk/ 
download/patient_power/ 
agreed_terms_conditions.doc). 


Health information content options 


There are a number of ways in which health 
information could be provided to patients at 
the bedside using the bedside television 
infrastructure. The options are fairly wide- 
ranging, and no comment is made here 
regarding the economic viability of the 
different methods of distributing information. 
The study merely looked at what was possible 
using the infrastructure that is being installed. 
This was based on the feedback provided 
through the interviews with those supplying 
the information, those using the existing 
services and those who aspire to be the 
content providers on these new systems: 

(1) A Health information television channel 
could be created running a wide range of 
health programmes, most likely across the 
spectrum of health subjects. As most 
people are only in hospitals for a few days, 
it would not have to provide new 
programmes every day, but it could be 
run with a high repeat ratio. There are a 
number of scheduling options for a 
service like this. This could lead on to the 
provision of a more widely available 
extended version for viewing as a digital 
television channel available at home. 

(2) A number of separate subject-specific 
health information channels could be 
created (e.g. Care of the elderly, Cancer, 
Maternity services, etc). Each “channel” 
could consist of a few hours of TV 
programming that would run on a 
continual loop (see section on “Bounty”, 
p. 342). All the channels could be 
available to all patients, or certain 
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channels could be made available only to 

beds or wards where the content is more 

likely to be relevant. 

Clips of videos or specific health 

information television programmes could 

be made availabie “on-demand”. 

Individual pieces of video could be played 

when specifically requested by an 

individual patient. If this were to consist 
of more than just a few short videos that 
are played sequentially, then a navigation 
system would be required to enable 
people to find the information for which 
they are looking. Depending on the 
uptake of this service, the systems could 
encounter bandwidth or contention 

issues. The cost of deploying such a 

service would be relatively high. 

(4) Patients are given access to a range of 
articles and leaflets as text that can be 
read on the television screen. The 
navigation would be in every respect like a 
Web site and could provide links to 
hundreds of pages of content. This 
content could be stored on a computer 


3 


server locally in the hospital or link out to: 


the Internet (or be a mix of the two). 

(5) A hybrid of (3) and (4) above, providing 
access to a mixture of text and video. 

(6) The health information service could be 
provided as an audio-on-demand service. 
So instead of (or as an alternative to) 
providing long articles for people to read, 
you provide a facility that reads the article 
to the patient. This could be as streamed 
audio or via the telephone (so along side 
each article a telephone number could be 
listed). 

(7) A health information radio service could 
be created. This could be through a 
locally delivered service, possibly created 
by providing resources, content or other 
support to the many hospital radio 
services, or as a national health 
information radio station. A national 
health information service could be 
distributed via digital television and made 
available both in hospitals and to home 
listeners. Radio has a number of 
advantages in that it is available in 
hospitals of all sizes (and so not reliant on 
the bedside TV infrastructure), it is low- 
tech and universally understood, it is low 
effort (you can listen with your eyes 
closed) and it is generally taken for 
granted that there will be a radio service 
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in-hospital. It could act as a “sustaining” 
service for local hospital radio stations. 
The bedside TV infrastructure will 
enhance access to radio services, and 
those patients will also have direct access 
at the bedside to a telephone enabling 
them to participate and contribute to the 
radio programmes. It would be 
considerably less expensive to create and 
run than a television service and could be 
more effective. 


All of these are options that exist when 
considering the provision of health 
information to patients at the hospital bedside 
via the television and telephone infrastructure 
being put in place. There are cost 
implications associated with each of the 
various options described, in terms both of 
content costs, plus additional local 
infrastructure costs for some options 
(particularly (2), (3) and (5)) and service 
management costs imposed by the bedside 
TV licence holder. They may also seek 
compensation, if it can be shown that the 
introduction of clinical services reduces 
patient demand for paid-for services. 


The source of health information content 
There is a great deal of duplication of effort 
in the creation of content for existing patient 
health information resources. The majority of 
existing resources are put into the creation of 
patient leaflets, which may then be made 
available via the Trust Web site and intranet. 
The text for these leaflets is sourced from a 
range of places, including NHS Direct 
Online, the Trust’s own resources (producing 
leaflets authored by their own health 
specialists, communications people, PAL 
service, research or library resource), or 
through a relationship with a commercial 
health information provider. 

Efforts that have been made to try and 
reduce this duplication include the provision 
of resources such as NHS Direct Online. This 
content is available without cost for Trusts to 
use within their own Web sites or intranets, 
and can also be incorporated into patient 
leaflets. Despite content being available 
without cost, uptake of information resources 
like this does not appear to be so high. 

The reason for the low level of uptake and 
use of central information resources is likely 
to be a combination of factors: 
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¢ Awareness of its existence may be low. It 
is certainly true that some commercial 
information providers are much more 
proactive in marketing their services to 
individual Trusts. 

* The lack of a single person within each 
Trust with responsibility for patient 
health information may have also 
hindered the communication process. 
The introduction of a PALs service 
should improve this (although co- 
ordination of the financial resources 
required for patient information is not 
held by any one department within a 
Trust, so in order to create a patient 
health information leaflet money has to 
be “found” from a variety of different 
pots). 

* Regardless of the source of the content 
for any patient information service, 
management time is still required to co- 
ordinate the process. 


For a price, private sector organisations will 
provide a managed service that is tailored to 
the specific information needs of that Trust, 
thus reducing the management resource 
required. A Trust, or individuals within the 
Trust, may not agree with the style of content 
from central resources such as NHS Direct 
Online; a private sector information provider 
will help to adapt content to suit local 
requirements, while also taking responsibility 
for any liability risks associated with the 
provision of information. 

It is explicit within the NHS Plan that 
patient choice in the provision of health care 
should be extended, so it would seem logical 
that patients might expect also to have a 
choice of different health information sources. 
The television infrastructure will be a 
powerful tool, if clinicians of all sorts and all 
grades support the ability of patients to access 
health information at the bedside. If it saves 
the clinician time, then they will recommend 
that a patient use the system -to obtain 
information. If it then helps patients to better 
understand what is going on around them, it 
is a win-win situation. 

In addition to the interactive television trials 
funded by the DoH between 2000 and 2002 
(a service that will roll out nationally in 2004), 
three other health information television 
services already exist in the UK: 

(1) Bounty. A direct marketing and sampling 
company that has specialised in the 
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provision of information and product 
samples for expectant and new mothers. 
They reach 95 per cent of expectant 
parents and 97 per cent of all new mums 
with their information guides and packs 
of free samples. The company produces a 
wide range of health information guides 
for parents, and for‘ health professionals, 
including a range of videos for midwives 
and parent educators, which have 
recently been updated. Bounty has been 
discussing making part of this updated 
video material available only in maternity 
units of hospitals via bedside TV services. 
This would be offered to patients as a 
separate television channel, repeating the 
same programme content several times 
each day. 

(2) QTV Rotherham. QTV is a non- 
commercial service that is funded by the 
local Primary Care Trust and has been 
operational for six years, providing a 
health information television service to 
local people and patients in the 
Rotherham area through a network of GP 
waiting areas, out-patient surgery waiting 
areas, hospital waiting areas, pharmacies 
and some specialist libraries. There are 
now around 50 sites that show a 30- 
minute video in these public places (all in 
the Rotherham area). Content is made up 
of short programmes or features (usually 
four to seven minutes long, but — 
sometimes less than a minute). 

(3) Channel Health Direct. This is a 
commercial television service available in 
the waiting-rooms of 1,200 GP practices 
in the UK. Several hours of television 
programming is shown in each GP 
practice on a continual loop, with 
updated material delivered via installed 
satellite equipment every few days. 


Electronic health information providers 


There are millions of health-based or related 
Web sites, and hundreds of respectable health 
information providers, varying in quality from 
the brilliant to the absurd. (Absurd example: 
www.curecancer.com, where you will find the 
causes of all cancers explained, and details of 
how to cure them as well.) Robert Kiley 

provides a number of absurd examples in an 
article “Quackery on the Web” (Kiley, 2000). 
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- Anumber of organisations have commercial 
relationships with one part or other of the 
NHS, while others are funded by the NHS or 
other publicly funded bodies. For example: 

e InTouch with Health 
(www.surgerydoor.co.uk) operate a suite 
of commercially funded patient 
information services, available via touch- 
screen kiosks and the Internet, and have 
an existing relationship with 64 Trusts in 
the UK (in 170 sites). 

e Medic Direct (www.medicdirect.co.uk) is 
a commercial company, with content 
provided by over 90 NHS teaching 
hospital consultants and GPs. 

* EQUIP - Electronic Quality Information 
for Patients (www.equip.nhs.uk) acts as a 
quality filter for patients looking for 
health information on the Internet by 
signposting them towards “good quality” 
sources of information, including those of 
the relevant patient support groups on a 
range of health topics. It is funded by the 
Public Health department in the West 
Midlands: 


The Internet 


All the main providers of the various bedside 
television systems are able to provide access to 
the Internet via their later generation bedside 
TV systems. As of September 2002 only one 
hospital (High Wycombe) was a “live” 
example of an Internet service, although other 
sites have since become operational. 

By the year 2005 the vast majority of the 
installed bedside television systems will 
enable patients to access content from the 
Internet. ‘The efficiency of the systems for 
doing this is not known. In most cases the 
systems intend to use slim versions of a 
Microsoft Internet browser, and there are 
significant differences between the ways in 
which the different systems will process data 
to be displayed on the browser. In some cases 
this may mean that a search of just a few Web 
sites could be a relatively slow process. 

Speed of access could be enhanced by 
developing versions of content specifically for 
the different systems and browsers in use. 
This could. also include agreements with the 
Trust or bedside television service provider 
for content to be cached locally at each 
hospital site, particularly for any rich media 
such as video, audio or graphics. Some Trusts 
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have expressed concern about patients having 
uncontrolled access to the Internet from their 
hospitals beds, particularly young or 
vulnerable people. Others take the view that 
they have no right to restrict what people are 
able to see. The option exists to offer access 
only to specific “approved” Web sites, or 
indeed only to an intranet (described as a 
“walled garden” of content in TV parlance). 
The other option would be to put some form 
of “nanny” software on to the system that 
would block access to undesirable Web sites 
(such as those that include pornography). All 
the main three suppliers intend to make use of 
slim versions of Microsoft browsers for 
Internet use and for navigating locally held 
information content. It will be possible to 
stream media (audio and video). 


Television 


Text that is to be viewed on a normal 
television must be displayed very carefully, 
especially as people viewing television at 
home can sit as much as ten to 15 feet away 
from the screen. Small text is difficult to 
display clearly and at that distance cannot be 
read anyway. In contrast, the TV service for 
hospital patients is really a slim-line computer 
monitor that is connected to a computer 
network. The screen is likely to be viewed at a 
distance equal to the length of the patient’s 
arm (at a similar distance to that at which 
someone sitting at a desk would view a 
computer screen). The television equipment 
being installed is mounted on a flexible arm 
that is fixed to the wall or ceiling, that can 
easily be moved closer or to a comfortable 
viewing position, or out of the way when not 
in use. Lack of space is a key factor, and a 
major reason why a flat screen technology has 
been deployed. Some of the operators intend 
to provide touch-screen units. 

There is an interesting shift in perception 
that occurs, once it is realised that the bedside 
TV services are actually computer monitors 
operating on a local computer network — 
rather than.just as televisions. The television 
signals are distributed to each bedside 
through a network of cables. Although the 
design or architecture of each system is 
different (for each of the different licence 
holders), in all cases the infrastructure being 
installed is in effect a local computer network. 
The software (middleware) that is available 
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on a television set-top box that one might 
have at home to receive Sky or Cable TV is 
used to interpret the interactive date, but this 
is not available on the bedside television 
systems. It is, therefore, not possible to access 
the interactive services offered by existing 
digital television providers (although some do 
provide access to some tele-text-type 
services). 

None of the first generation services 
provides direct access to the Internet (some 
provide a jack-point, where a laptop or other 
device with a modem can be plugged in to use 
the telephone line). All the latest generation of 
units from the main three licence holders 
provides access to the Internet. There may be 
contention issues (i.e. restrictions on the 
number of people who can access information 
at the same time without the “network” 
slowing down). None of the systems will allow 
patients to download material (so e-mails will 
not be able to have attachments). There will 
be firewalls in place to prevent this. 


Content in the “hospital information” 
section of bedside TV services 


The structure and content within a hospital 
information section depend on the 
enthusiasm of the implementation team, and 
the priorities of the Trust. In a limited 
number of Trusts, one or more videos about 
the hospital are available (e.g. in 
Southampton, where the bedside TV service 
supplier is Patientline, a number of videos are 
available via the system). i 

Once a patient has logged on to the 
Patientline system, an introduction video 
from Patientline is played to them, followed 
by an introductory video from the hospital — if 
there is one; otherwise text about the hospital 
can be displayed. Medical information or 
content can be seen as a problem because of 
the need for it to reflect the policy of that 
Trust, or the views of the senior clinicians in 
that field from that Trust. In most cases, at 
the moment, that initial plan is to provide just 
basic information about the hospital and the 
services provided, in text form. Like a Web 
site, this could be a single page or be 
menu-driven. Quite a number of Trusts are 
interested in electronic menu-ordering 
systems and a lesser number have been 
looking at the options for using the system for 
patient surveys. 
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HTS have been looking at-a medical Web 
site that could feature as a sub-section of the 
hospital information channel, providing 
health information. This site features a 
substantial amount of video material. The 
intention would be for the video content to be 
cached locally within the hospital, but for text 
to be accessed via the Internet (the name of 
that content provider was not available at the 
time of the interview). 

Patientline research in 2000 indicated that 
patients want to have access to information 
about their condition, as it rated over and 
above access to television or movies. 


Other potential uses for the bedside TV 
systems 


Access to pharmacy services was discussed by 
all the suppliers, in particular the ordering of 
tests, obtaining test results and some form of 
electronic prescribing. One supplier also 
discussed the potential for the integration of a 
pharmacy service with a system that makes 
checks on potential conflicts with other 
medication being taken by that patient. Also 
of interest is the ability to use the bedside TV 
service to access some form of clinical 
decision support software. 

Northwick Park Hospital has been looking 
at using the infrastructure to introduce a new 
security scheme. The idea is to have a unit in 
the hospital reception. Visitors would enter 
the patient’s telephone extension number and 
a photograph of that visitor would be sent to 
that patient, who could say yes or no to their 
being allowed on to the ward. 

All of the systems can be used in 
conjunction with smart cards, that could be 
used as a micro-payment system for other 
services (to pay for things from shops or 
commercial service providers within or local 
to the hospital). 

Another area of interest is the management 
of admissions through the bedside TV system, 
as well as patient discharge from hospital 
(which would lead to more efficient bed 
management throughout the hospital). 

All the systems provide free access to a 
number of radio channels, including the local 
hospital radio station. Some also provide 
access to talking books. These can be 
accessed like a radio channel, with a number 
of set starting-times each day. One service 
provider mentioned that they provide access 
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to a number of games, but admitted take-up 
was relatively low. 


Surveys 


All the suppliers said that their systems could 
be used for patient surveys. Wandsworth 
indicated that there had been little interest in 
this functionality at this stage, with the 
priority being to get core services operational. 
Patientline reported that their first generation 
system is already used for patient surveys and 
that their latest system has enhanced survey 
capability and the first such system to be used 
for this purpose will go live shortly. The 
system could be used for “national” surveys, if 
supported by the local Trusts. 


Electronic health records 


The digital systems of all three suppliers are 
capable of providing access to information 
that is held electronically. Security is a vitally 
important issue in this area, and one that all 
the suppliers claim to have resolved. There 
are three uses. First, where the clinician will 
take control of the screen and use it to check 
something, but not share the view with the 
patient. Second, where the view of 
information on screen is shared between the 
patient and the clinician. Third, where 
patients have access to some part of their own 
health record. 

A number of Trusts discussed their desire 
to be able to use the bedside TV systems to 
provide patients with access to information in 
different languages. Others talked about using 
the service as part of the Trust’s informed 
consent procedure, potentially providing 
access to X-rays (although it is recognised 
that the resolution and definition are not 
suitable for diagnosis), as a tool for reference 
and explanation to a patient. 

There is also the potential for using the 
service for health promotion purposes, such 
as the provision of rehabilitation material for 
patients who have suffered a heart attack, or 
for providing new mums with details about 
breast-feeding, or showing people suitable 
post-operative exercises. Personal reminders 
could be issued to remind individuals when 
they need to take medication or of 
recommended exercise routines. 
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Conclusion 


There is potential to use the bedside TV 
infrastructure being installed in hospitals to 
provide patients with access to health 
information. In what form it will be made 
available will be influenced by a combination 
of who is paying for the content, who is 
providing the content and the economic 
model that has been established for the 
commercial operators of the infrastructure. 
Their priority is to maximise the revenue they 
receive from each bed. 

The DoH will deploy an interactive health 
information service on the main digital TV 
platforms (cable, satellite and terrestrial) in 
2004. Although it would be logical to look to 
this service to be extended on to bedside 
television in hospitals, this is unlikely to be 
possible. More likely is it that the bedside TV 
service will provide patients with access to the 
Internet. , 

Currently the content for any information 
services provided on the bedside TV service is 
agreed at a local level by the Trust concerned, 
but for this medium it is likely that decisions 
will be led by the infrastructure supplier, who 
may have preferred supplier arrangements in 
place. What is not clear is what sort of 
appetite patients have for consuming and 
absorbing health information while in a 
hospital bed, and what difference it makes if 
the information is provided in text, audio or 
video form. One might assume that video 
content had the greatest potential to be 
engaging and to clearly convey messages and 
information, but the cost is much higher. 
Radio services are an integral part of the new 
infrastructure, but rarely mentioned. When 
discussed with a number of Trusts, the 
consensus was that patients took it for granted 
that there would be access to radio while in 
their hospital bed. This led to the thought that 
radio might be a more effective and efficient 
way of providing people with access to health 
information than text or using videos. 
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Areas that would be suitable for further 
research 


* What are the effects of patients having 
direct access to comprehensive and high 
quality health information at the hospital 
bedside on their relationship with medical 
professionals in the hospital? 

+ Are patients receptive to health 
information when they are already in a 
hospital bed? 

* Does an informed hospital patient require 
more individual consultation time with 
clinicians or less time? 

* By providing patients with access to a 
bedside television service, and the 
Internet, are individuals more likely to 
make use of other electronic or online 
services elsewhere in the future? 

+ What is the most effective way of 
providing patients with health 
information using the bedside TV 
infrastructure? 

¢ What is the potential for using radio as a 
channel for the provision of high quality 
health information for patients (via the 
bedside TV infrastructure)? 

* Should the NHS be a producer and 
“publisher” of health information, or act 
as a provider of advice and guidance on 
the best sources of high quality 
information? 
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Introduction . 


In the UK, Gisela Stuart, the (then) 
Parliamentary Under-Secretary of State for 
Health, announced in June 2000 that the 
Department of Health would fund a series of 
pilot projects exploring possible health 
applications of digital interactive television 
(DiTV). The aim of these projects was to 
provide patients with easy and fast access at 
home to health advice and information. 

This launch of NHS Digital, a development 
under the NHS Direct initiative, represented 
a further stage in the Government’s 
commitment to provide faster and more easily 
accessible information about health, illness 
and the NHS, so that people are better able to 
care for themselves and their families. While 
the Internet has so far been the primary 
conduit for provision of “online” health 
information, the need to reach demographic 
groups in society who have not yet signed up 
to cyberspace, another older technology has 
entered the frame. Television is the most 
ubiquitous mass medium and has evolved 
technologically in recent years to the point 
where it may become an alternative 
interactive medium to the PC-based Internet. 

The 1997 White Paper, The New NHS 
(Department of Health, 1997), identified the 
Internet and digital television as key media 
through which public access to the NHS 
could be improved. It was envisaged that new 
interactive communications media could 
empower patients to take more care of their 
own health as well as improving the efficiency 
of health provision in the UK. 

In this exciting new world of modern 
interactive communications technology, 
digital television has been regarded as having 
greater potential than the Internet because 
television is already a well-established 
medium. Nearly every household in the UK 
has at least one TV set and, despite the 
relative success of supermarket online 
shopping, more people feel comfortable using 
TV than the Internet. Fewer people are likely 
to be excluded from a service that is available 
via television than from one that can only be 
accessed via a computer. More people may be 
persuaded to engage with interactive 
information services with the more familiar 
TV remote control than with a PC mouse. 

The Department of Health contracted four 
consortia to experiment with a variety of 
interactive services through which registered 
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users can access large banks of health 
information via their TV sets. Users were able 
to navigate this online health encyclopaedia 
using a standard digital TV remote control of 
the kind typically issued to new digital 
satellite, cable or broadband subscribers. In 
addition to largely text-based information 
services, the pilots included a number of 
transactional services. These allowed viewer- 
users to make appointments with their doctor 
or with specialised health services, volunteer 
their own services to community health 
support groups, maintain online medical 
records, or obtain a live video link to an online 
NHS nurse, who appears on their TV screen, 
while talking to them over the telephone. 
While studies have emerged of the use of 
online health information services provided 
through the Internet (Eysenbach and Kohler, 
2001; Pennbridge et al., 1999; Quintana 
et al., 2001), the current research was the first 
to examine the provision of such information 
via interactive television. This paper will focus 
on the Living Health service that was the first 
of the four to launch and to complete its pilot 
run between June and December 2001. The 
analysis reported here focuses on use of the 
text-based information service provided by 
this channel. 


The Living Health service 

Living Health offered two types of service: 
Interactive and Transactional. The 
Interactive Service was a content database on 
a wide range of health topics and issues. The 
Transactional Services comprised video 
access via the TV set to an NHS Direct nurse 
called up over the telephone and a facility to 
book appointments with a local GP. 

Most of the text remained unchanged 
throughout the duration of the pilot. Updated 
content was provided on Health News 
(updated daily), Medicines Directory, Local 
Services Directory (updated monthly), and 
Public Health Alerts (notified by Birmingham 
Area Health Authority). 

The general text-based interactive service 
was Officially launched, along with the GP 
appointments booking service, on 28 June 
2001. The In-Vision Service was initially 
rolled out across Birmingham cable hubs 
from 30 July 2001, but the official launch of 
this service did not take place until 3 
September 2001. 
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Methodology 


Research on all DiTV consortia adopted a 
multi-method approach that included digital 
log analysis of real-time user behaviour, 
supplemented by ad hoc self-completion 
surveys and in-depth interviews to gauge 
users’ reactions to the services. This paper 
will focus on the results from the log analyses 
and a survey of viewer-users of Living Health. 
This pilot yielded the most comprehensive 
data on users of the four consortia. 


Log analysis 

Digital log data were supplied in electronic 
form by Living Health. These DiTV logs were 
relatively refined and accurate and compared 
favourably in these respects with Web logs. A 
major problem in analysing log data is that of 
“caching”. Pages viewed on DiTV were 
cached, as typically are Internet pages 
uploaded to the end-user’s PC. “Caching” is 
an Internet browser feature switched on at the 
user’s machine and means that pages, once 
viewed, are available from the terminal being 
used. Thus, any pagés re-viewed do not have 
to be downloaded again from the server, 
saving considerable time. 

In respect of recording user behaviour via 
logs, however, this practice creates a problem. 
“Views” of previously seen pages are made 
from the cache and are therefore not recorded 
by the server access log file. The requested 
cable DiTV pages used in this study were 
routed through hubs that were then sent on to 
the user’s TV set. Once a page had been 
requested, the hub cached the page and made 
it available to other users on the hub without 
re-requesting the page from the original 
server. Again, this resulted in an under- 
reporting of pages viewed. Fortunately, 
Living Health employed a method to obviate 
the effects of caching. The process was 
defeated by the inclusion of a non-cacheable 
“gif”. The idea here was that each page 
included an image that cannot be cached by 
the hub and so requests are made to the server 
every time and a page count recorded. 

Despite initial pessimism over provision of 
user identification ‘data, individual subscribers 
could be identified via the log data supplied, 
but in this context “subscriber” referred to a 
household rather than to an individual 
person. Hence, “user”, in the sense employed 
here, may refer to a family of users. 
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Log data enabled the measurement of 
overall numbers of users, and changes in these 
numbers over the pilot period. Usage data 
could be displayed in terms of the raw 
number of users using the service per day, the 
reach, and return visitor (and the numbers of 
users making return visits). Further measures 
included the length of time spent tuned in to 
the service and the number of pages or 
screens viewed during a visit. These data 
could be further broken down by topics 
viewed. Hence, it was possible to ascertain if 
some parts of the service were more heavily 
used than others. Log data were analysed over 
a period of 20 weeks running from the end of 
June to November 2001. 


Users’ perceptions of the DiTV service 
A questionnaire survey was distributed with 
Living Health promotional material to 38,000 
Telewest cable subscribers in Birmingham, 
halfway through the pilot period. In all, 723 
usable questionnaires were returned (2 per 
cent of the population of Telewest cable 
subscribers in this area). 

The questionnaire was designed primarily 
to obtain responses on the use and non-use of 
digital television for health information, but it 
also asked respondents to rank the 
importance of a variety of other health 
information sources, including health 
professionals, people known to the 
respondent, health literature, mass media and 
other online media (e.g. telephone help-lines, 
the Internet). Respondents were asked to rate 
the health benefits of the Living Health 
service, especially with regard to whether it 
had helped them to understand a particular 
health condition better or helped them in 
their dealings with their doctor. Respondents 
were asked if they found the interactive 
service easy to follow and to rate the 
comprehensibility of the information 
supplied. Ratings were also obtained of the 
perceived importance of the health topics 
covered by the service. Personal details were 
obtained from respondents about their sex, 
age, occupation and whether they have 
responsibility for the health of another person. 

More females (53 per cent) than males (46 
per cent) replied to this survey. There was a 
gocd age spread among respondents, with 
those aged 36-45 (24 per cent) and 26-35 (22 
per cent) being most likely to respond. 
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Results 


Daily use of health information on TV 
The actual number of logged Living Health 
viewers varied quite considerably over the 
survey period. The text information service 
was launched on 28 June 2001. Just after the 
launch of the channel it stood at just under 
400 users a day. On 3 August, the service was 
hit by technical delivery problems that 
restricted service to just two of its four server 
hubs that delivered the information locally. 
The service resumed fully on 23 August. The 
number of users remained at approximately 
320 users a day, for the month of September. 
The number of users then fell after 24 
September and reached a low of 200 on 9 
October. From the beginning of October, the 
number of daily users remained in the band 
between 220 to 270 until the pilot period 
ended. The reason for the relatively sharp fall 
towards the end of November is unknown. 
Use at the beginning of the pilot period (up 
to 14 July 2001) stood at approximately 
14,000 page views a day. From September, 
use fluctuated within the range between 7,300 
to 7,800 daily page views. There was, 
however, a sharp and as yet unexplained 
decline in use in the last week of November. 


Return visitors 
Repeat visits to a site constitute conscious and 
directed use and a possible indicator of site 
appeal. A service with a high percentage of 
returnees can be regarded as having a “brand” 
following, which is what all service providers 
aspire to achieve. All this makes “return 
visits” a powerful performance indicator. It 
seems logical that the more times a user 
returns to the site, the more likely they are to 
be satisfied with the site. Nearly six in ten (59 
per cent) users visited the service just once in 
the survey period, with four in ten (41 per 
cent) visiting the service more than once. 
Further analysis showed that those users 
who visited the site early after its launch were 
more likely to be repeat users. Looking at 
Figure 1, it can be seen that users who first 
visited the service in July/August on average 
visited the channel three times during the 
remainder of the pilot period or about once a 
month. In looking at just those users who had 
visited twice, the average number of days 
between the first and second visit was 
calculated at 26 days — and also suggested a 
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Figure 1 Number of times users visited digital health TV by date of first 
visit 
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return visit pattern of approximately once a 
month. 

Generally, as the number of visits 
increased, so did the time spent viewing 
(during a session), although session length 
was shorter again for those users who visited 

¥ the site over 15 times in the period of study. 
Figure 2 shows the session duration by the 
grouped number of times users had visited. 
Users who had visited once only had the 
shortest session time of approximately two 
minutes, while users visiting six to 15 times 
recorded the longest session durations of 
about seven and half minutes. Those who 
visited the service over 15 times had a 
shorter average session time of 
approximately six minutes. 


igigure 2 Number of times users visited digital health TV by estimate of 
(median) session length 
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Reach i 
Most people are unlikely to check health 
information, even through a convenient 
online source, every day or even every week, 
unless they have a special reason for doing so. 
It is more likely that health information 
seeking will be conducted on an ad hoc basis. 
Average rates of site visits represent one type 
of metric that indicates the impact a digital 
health site has had upon a market. Given the 
likely pattern of behaviour associated with 
such services, however, it is important to 
know how many potential viewers or users 
have tuned in at some point or another. This 
information is provided by the reach metric. 
The Living Health DiTV service was 
available to between 35,000 to 40,000 
potential users over the duration of the pilot. 
These were households that subscribed to the 
cable TV network over which the service was 
carried. The subscriber figure fluctuated over 
time, reflecting the constant churn (i.e. loss of 
existing subscribers) and recruitment of new 
subscribers. Log data indicated that between 
34 per cent and 39 per cent of potential users 
accessed the service during the pilot period. 
Projecting this figure to the overall cable 
network population, it was estimated that, 
during the 20-week period monitored, 13,718 
different visitors (i.e. subscribing households) 
used the system. 


Depth of visit 

This measurement was indicated by page 
view durations, session durations and 
numbers of pages viewed per session (see 
Table I). Using Huber’s robust M-estimator, 
the average page view time was about 13 
seconds, session time was just over four 
minutes and the number of pages viewed in a 
session was about 15. 

The numbers of pages viewed is an 
important metric in that it indicates service 
penetration. The deeper into the service users 
are prepared to search, the more involved they 


Table | Average page view time, session duration and number of pages 
viewed per session 


Page view Session Number of 
time time pages ina 
. (secs) (secs) session 
Mean 16.84 421.11 23.62 
Median 12.00 248.00 13.00 
5 per cent trimmed mean 14.19 350.74 19.15 
Huber’s M-estimator 12.60 278.69 14.63 
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become. However, there is a further 
characteristic of the configuration of the 
DiTV health service that adds further 
meaning to this measure. The arrangement of 
the health information was hierarchical. Users 
were required to navigate menu screens 
before they reached actual information pages. 
Menu information might occupy the first two 
or three pages clicked on. It is important 
therefore to examine carefully data on 
numbers of pages viewed in the light of the 
type of information different pages contained. 

Over one in five (22 per cent) users of 
Living Health viewed one to three pages. 
These users were unlikely to have penetrated 
past the menu screens and would not have 
viewed an information page. A large 
proportion of users (78 per cent) navigated to 
an information page (by penetrating beyond 
the third page) and over one in three (37 per 
cent) showed considerable interest by viewing 
more than 20 pages on a visit. These figures 
proved to be fairly stable over time. 

Figure 3 presents data for median session 
durations in seconds by grouped number of 
pages viewed. As expected, the more pages 
users scanned, the longer was the duration of 
their sessions. Hence, users who looked at 21 
or more pages in a session had viewing 
duration of approximately ten minutes and 
users who looked at between one and three 
pages conducted sessions that lasted only 
around 30 seconds. 

Figure 4 presents page view time by the 
grouped number of pages viewed. Users 
viewing between one and ten pages had a 
page view time of approximately 14 seconds, 
but this fell to an estimated nine seconds per 


Figure 3 Session duration (median) by number of pages viewed 
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Figure 4 Page view time by the number of pages viewed 
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page for users viewing over 21 pages. Users 
who looked at more pages were more likely to 
be repeat users and were likely to quickly 
navigate through the menu pages. 


Topics viewed 

To reiterate the layout of this service, Living 

Health’s home page on DiTV offered users 
eight topic categories: Today’s Health News, š 
Healthy Living, Men’s Health, Women’s 
Health, Children’s Health, Illness and 
Treatment, Local Health Services, and NHS 
Direct InVision. The first seven categories 
offered routes through to underlying textual 
information on a wide range of health topics. 
The Local Health Services section also 
opened a gateway to the GP appointments 
booking service and contained information 
about working in the NHS especially targeted 
at NHS personnel or potential employees. 
NHS Direct InVision was the live, 
transactional video nurse service. 

Figure 5 shows the health topics people 
chose to view. Illness and Treatment was the 
most popular topic by some margin, with 
nearly four in ten users (39 per cent) viewing 
these pages. This perhaps points to the direct 
and practical application of digital health 
information on DiTV. 

Figure 6 shows the average page view time 
by section. The longest page view time 
(median) was recorded for the Preview 
section and LHAppBookings (the doctor’s 
surgery bookings service). This section È 
recorded approximately two and half times 
the average page view time: 28 seconds 
compared with 13 seconds. The other 
sections recorded very similar viewing times. 
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Figure 5 Health topics viewed 
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DiTV users showed a marked interest in 
sexually related health topics. The most 
watched pages on Living Health were 

+- dominated by content about sex and sexual 
problems. These included a range of 


specifically men’s and women’s problems and 


other issues associated with sexual 
orientation: l 

* orgasm problems; 

* impotence; 

* — prernature ejaculation; 

* keep your sex life in good shape; 
* no content; 

* NHS Direct InVision; 

: dyspareunia; l 
sexual infections; 

* gay sex; oe 

* sexual health help; 

e -thrush and cystitis; 

* preventing prostate cancer; 


Volume 55 - Number 5/6 > 2003 « 346-356 


* flatulence; 
* practising safer sex; and 
¢ injury treatment principles. 


The above topics accounted for 7.9 per cent 
of all information pages, and number of 
unique pages viewed was 2,798. 


User survey 

A clear majority of respondents (72 per cent) 
claimed to have heard of Living Health. 
Fewer than one in four (23 per cent) claimed 
to have used the service. Among users of the 
service, there were more women (57 per cent) 
than men (43 per cent). Users exhibited an 
older rather than younger age profile. Three 
in ten users (30 per cent) were aged over 55, 
over four in ten (43 per cent) were aged 
36-55, one in five (19 per cent) was aged 
26-35, and under one in ten (8 per cent) was 
aged 25 or under. 

While users comprised a higher percentage 
of women than men, this pattern changed 
with age. Men were in the majority among the 
56-65s (61 per cent) and the 66-75s (92 per 
cent). Women users were in the largest 
majority among the 26-35s (81 per cent) and 
the 16-25s (69 per cent). These gender and 
age profiles may indicate a growing concern 
about their health among older men and an 
interest among young women, triggered by 
the presence of children. l 

Among users of Living Health, over eight in 
ten said they found the service useful (81 per 
cent), easy to navigate (82 per cent), and easy 
to understand (82 per cent) all or most of the 
time. Multiple logistic regression analysis, was 
used to identify important factors that 
explained use of Living Health. 


Predictors of use of DiTV | 

A multiple logistic regression model was used 
to consider what factors could explain 
differences between those respondents who 
said they had used Living Health (coded as 
zero) and those respondents who had heard of 
the service but not used it (coded as 1). The 
model offering the best fit yielded three 
explanatory variables: these were whether the 
person had reportedly used NHS Direct in 
the last 12 months; how important to them 
the Web was as a tool for medical 
information; and whether the user had an 
interest in prescription drugs. Table II 
summarises the results. 
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Table II Predictors of reported use versus non-use of Living Health 


Variables in model 


Have you phoned NHS Direct in the last 12 months? 
No 
Yes 


How important is the Internet or Web for medical information? 


Not at all/not very 

Fairly/very 

An interest in prescription drugs 
Not at all/Not very 

Fairly/very 


N Log odds SE 
294 
145 0.35*** 0.22 
196 
243 1.44* (0.22 
119 
320 0.45** 0.26 


Notes: Comparisons made between respondents who had used the service and those who had heard of it but not 
used it; levels of significance (Wald’s Statistic): * p < 0.10, ** p < 0.05, *** p < 0.01; , I didn’t know about it: 
coded as missing; | have used it: coded as 0 (n = 138); | know about it but haven't used it: coded as 1 (n = 299) 





Respondents who had phoned NHS Direct in 
the last 12 months were much more likely to 
have used Living Health than those who had 
not phoned the NHS, confirming the earlier 
analysis. 

Respondents who considered the Web 
important as a source for health information 
were more likely to have heard of Living 
Health and not used it compared with those 
who did not consider the Web as an 
important source. This result suggested that 
those using the Web for their health 
information needs were not switching to 
digital television, even though they know 
about the existence and availability of this 
alternative service. For these people, the early 
DiTV appears not to represent a substitute for 
Web-based information. Further research 
needs to clarify whether these are distinct new 
media markets and likely to remain so. 

Respondents who reportedly had an interest 
in prescription drugs were less likely to have 
tuned in to this service than respondents with 
little or no interest in such drugs. Bi-variate 
analyses also indicated that respondents 
interested in prescription drugs were more 
likely to visit their doctor and more likely to 
be interested in information on a specific 
condition, both indicators of users who may 
be unwell. Of those who did reportedly use 
the Living Health service, approximately 55 
per cent said they had used the service either 
before or after a consultation with their 
doctor. Hence, the significance of 
prescription drugs in the model may well be 
an indicator of those users who were unwell. 
Another interpretation of this finding 
therefore might be that respondents who were 
unwell were more likely to be users of the 
Living Health service. 





DiTV and personal health issues 
Respondents who had used Living Health 
were asked how much help the information 
found had been to them in their dealings with 
their doctor, in improving their condition, in 
changing their feelings towards their 
condition and in understanding their 
condition. More than one in two respondents 
who had reportedly used Living Health (55 
per cent) said they had queried the system for 
information before and/or after a consultation 
with their doctor. A total of 49 per cent of 
reported service users said that it had either 
helped or helped a lot in dealing with their 
doctor. More than four in ten (45 per cent) 
said the DiTV information had helped them 
to improve their condition, more than one in 
three (35 per cent) said it had helped them to 
understand their condition, and one in three 
(33 per cent) said it had helped to change 
their feelings about their condition. 


DiTV as an alternative to visiting the 
doctor 

All respondents, whether they had reportedly 
used the Living Health service or not, were 
asked if they had used medical information 
that they had found from other sources as an 
alternative to seeing the doctor. This analysis 
permitted identification of the outcomes for 
those people using the service. 

The impact of digital information provision 
in other parts of the health service has always 
been on the minds of health managers and 
policy makers. A total of 98 per cent of DiTV 
users answered this question, of whom 53 per 
cent confirmed that they had used 
information found from other sources to 
replace a visit to the doctor. 
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A logistic regression model was used to 
explore which factors characterised those 
respondents reported to be more likely to use 
information in this way. Significant predictors 
were the practice nurse, NHS Direct phone 
line, and reported use of the Living Health 
service (see Table III). Newspapers, health 
books and magazines, the Internet, family or 
friends, and leaflets in the surgery did not 
emerge as significant predictors. 

The importance of information from the 
practice nurse exhibited a curvilinear 
relationship to using DiTV health 
information as an alternative to seeing the 
doctor. Respondents for whom the practice 
nurse was very important or not important 
were less likely than respondents who 
attached moderate importance to the practice 
nurse to use other sources of medical 
information instead of seeing the doctor. The 
finding among those who attached no 
importance to the practice nurse is not 
surprising and may indicate a group of 
respondents for whom the search for health 
information is not important at all. A different 
explanation is needed for respondents for 
whom the practice nurse is an important 
source of health information. With these 
respondents, it is possible that they prefer 
direct contact with health professionals. 

Respondents who had used an NHS Direct 
phone line were more likely than those who 
had not to say that they would use another 
source of medical information as an 
alternative to seeing the doctor. This may 
therefore indicate that this telephone help 
service can provide a functional alternative to 
seeing the doctor. 

The key finding for the current study was 
that respondents who had reportedly used the 
Living Health service were much more likely 
than non-users to say they would use other 
medical information sources as an alternative 
to seeing the doctor. 

Age and whether the respondent had a 
health responsibility for another person were 
significant as explanatory variables. 
Respondents aged between 56 and 75 were 
less likely to use information from another 
source as an alternative to a visit to the doctor 
than were all younger age groups. This 
finding was consistent with previous research 
on the use of consumer health information on 
the Internet (Nicholas et al., 2000). 

Health interest was related to who uses 
information as a substitute for visiting their 
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doctor. Health interest in the model was 
distributed over four variables — a general 
health interest, an interest in prescription 
drugs, an interest in healthy living, and an 
interest in alternative health. Those people 
very interested in healthy living were more 
likely to use medical information found from 
different sources as an alternative to going to 
the doctor than those very interested in 
alternative health and those interested in 
prescription drugs. Consumption of 
prescription drugs may be another indicator 
of a long-term health condition. Respondents 
with an interest in these drugs reported being 
more likely to use alternative sources of health 
information than to see the doctor. 


Discussion 


This study has indicated that DiTV can 
represent a viable health information platform 
and reach users who do not yet surf the 
Internet. Real-time behavioural use of this 
service was measured by digital log data that 
automatically provided an electronic 
recording of viewers’ visits to this online 
health information site via their TV sets. 
Then, three months into the 20-week pilot 
project, users were surveyed for their 
reactions to the service. 

Levels of daily use of the service fluctuated 
over the pilot period. After a brisk start, use 
dropped off (in part owing to a technical fault 
in data recording) and then recovered three 
months on. The numbers of daily users 
averaged in the hundreds, while available 
cable TV subscribers numbered in tens of 
thousands, but, over the duration of the pilot, 
it was estimated that around one-third of 
subscribers to this metropolitan cable network 
had tuned in to the digital health service at 
least once. Furthermore, early users were 
most likely subsequently to become repeat 
users, though average repeat usage was about 
one visit to the site per month. The Living 
Health service offered a large quantity of 
content on a wide range of health topics. The 
most frequently sought-after information, 
however, was that concerned with sexual 
problems. In this respect, DITV differed from 
other online health information sources such 
as the Internet and touch-screen kiosks, 
where the most popularly searched health 
topics concerned heart disease, backache and 
asthma (Nicholas et al., 2001). 
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Table II! Predictors of use of sources of medical information as an alternative to seeing the doctor ` 


Variable in model N Log odds ratio SE 
How important is the practice nurse for health information? 

Very important 270 

Fairly important 208 2.08*** 0,22 
Not very/at all 97 1.41 0.29 
Have you phoned NHS Direct in the last 12 months? i 
No 394 

Yes 181 1.46 0.22 
As a Telewest subscriber, have you ever heard of a service 

called “Living Health”? 

I didn’t know about it 143 

I have used it 136 1.84** 0.29 
l know about it but haven't used it 296 0.96 0,23 
What is your age group? 

Under 25 46 

26-35 141 0.82 0.40 
36-45 156 0.72 0.40 
46-55 94 0.95 0.43 
56-65 77 0.41** 0.45 
66-75 44 0.37** 0.50 
Over 75 17 0.87 0.65 
Do you currently have responsibility for the health of another? 

No > 281 

Child(ren)} 174 1.35 0.25 
Other ~ family 64 0.61 0.31 
Other/multi-carer 56 2.56"° 0.36 
How interested are you in information on health topics? 

Very interested 230 

Quite 281 0.69* 0.21 
Not very/at all 64 0.37*** 0.39 
How interested are you in prescription drugs? 

Not very/at all 155 ; 

Fairly important 251 1.58** 0.24 
Very important l 169 1.52 0.28 
How interested are you in healthy living? 

Not very/at all 56 ; 

Fairly important 250 1.81* 0.38 
Very important 269 2.96*** 0.41 
How interested are you in alternative health? 

Not very/at all 189 

Fairly. important. 221 1.69** 0.24 
Very important 165 2.50*** > 0.28 


Notes: Levels of significance (Wald’s Statistic): * p< 0,10, ** p< 0.05, *** p < 0.01; No: coded 0 {n = 248); 





Yes: coded 1 (n =.327) ` 


While logs can provide comprehensive data 

on real-time behaviour, such data are purely 
descriptive and reveal little or nothing about 
the reasons for using the service. To find out 
more about this, users were surveyed for their 
reactions to Living Health. Just under one in 
four survey respondents reported tuning into 


this interactive health information service 
through their TV sets. These viewer-users 
were predominantly likely to report 
favourable experiences with the service in 
terms of its basic usability and usefulness. 
Users were distinguished from non-users 
especially with regard to whether they had 
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telephoned a national health help-line in the 
previous year, regarded the Internet as an 
important source of health information, and 
had an interest in prescription drugs. The 
significance of Internet use here, though, was 
that it predicted greater non-use of the DiTV 
service. This finding contrasts with earlier 
research that indicated the importance of 
compatibility between old and new 
technologies in technology adoption (Ettema, 
1984; Dutton et al., 1987; LaRose and Atkin, 
1988). Adoption of telecommunications 
technologies, for example, was most 
powerfully predicted by use of other similar 
technologies (Reagan, 1987; Lin, 1994). The 
current research has indicated that DiTV 
does reach more elderly, lower socio- 
economic status and under-privileged people 
(in particular those with young children) 
compared with the Internet. 

Online health information via DiTV was 
found to serve as a valuable alternative health 
information source to their doctor for many 
users. Digital health information on TV could 
be used to double-check information supplied 
by the doctor or in preparation for a 
consultation to supply questions the patient 
might ask the doctor while in the surgery or 
hospital. These findings confirm responses of 
online medical and health information users 
in the USA. Research conducted under the 
auspices of the Pew Internet and American 
Life project has shown that half of online 
health information users (50 per cent) 
claimed that Internet information led them to 
ask a doctor new questions or to get a second 
opinion from another doctor. Over one in 
four US Internet health information seekers 
(28 per cent) said that Internet information 
affected their decision about whether or not to 
visit a doctor (Fox and Rainie, 2000). 

Although the return sample was small by 
comparison with the contact population, with 
well over 700 respondents, it can be regarded 
as robust enough to provide meaningful initial 
indications about the potential that DiTV 
may have as a health information platform. 
There are several findings that stand out in 
this regard. 

First, the demographic profile of viewer- 
users indicated that DiTV, as an interactive 
technology, is likely to be used by a wide 
social spectrum. More significant, however, is 
the finding that it attracts a different profile of 
users from the Internet. In particular, one of 
the key predictors of service use was income 
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of household, with users from low-income 
households comprising a prominent user 
group. 

Second, there is a clear functionality to 
DiTV health information. Again the user 
profile indicates a strong presence of older 
males (55+) and young adult females. The 
older male profile represents the clear 
presence of a group who become increasingly 
concerned with health matters, in a gender 
that is normally reluctant to seek medical 
diagnosis when younger. This conclusion is 
supported by the emergence of interest in a 
health condition and use of prescription drugs 
as significant predictors of DiTV use. The 
young adult female profile reflects the health 
concerns of young women that may often be 
child-related. This observation is reinforced 
by the result that households with children up 
to the age of four were among the major users 
of DiTV health information. 

Third, DiTV opens up an additional and 
complementary platform for health 
information to the Web. Even more direct 
evidence of this than the demographic profile 
of DiTV users was derived from the finding 
that survey respondents, for whom the 
Internet was an important source of medical 
information, were much more likely to be 
non-users than users of the Living Health 
service. For DiTV users, the Internet was 
seen as a relatively unimportant health 
information source. 

Fourth, the functionality of DITV in a 
health context came through from reports of 
the way users utilised its health content. Most 
users claimed that DiTV health information 
was consulted in connection with a doctor’s 
appointment. Around half of these DiTV 
health information users said that it had 
helped them in their dealings with their 
doctor. Further evidence of the value of DiTV 
as a health information source came from 
analyses that showed that respondents who 
said they used other sources of medical 
information as an alternative to seeing their 
doctor were especially likely to be users of the 
Living Health service. In this context, DiTV 
health information ranked alongside 
consulting a practice nurse and using NHS 
Direct. Notably, use of the Internet did not 
emerge as a significant predictor of this 
behaviour. 

The Department of Health in the UK has 
made a major investment in testing the 
potential of DiTV as a health information 
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platform. The early indications are that this is 
a platform with promise. Longer-term 
commitment and observation will indicate 
how effective this technology can be in 
relation to a range of different health 
information and advisory functions and how 
best it can be utilised to benefit public healzh 
care and the health profession. 
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Abstract 


The subject of this study is the availability of, and access 
to, information concerning individuals’ end-of-life health 
situations. The sensitive issues surrounding this subject 
are considered in detail, and the restraints, primarily legal, 
placed upon information availability highlighted. 
Questionnaires and interviews were used to determine 
public and professional awareness of, and attitudes 
towards, the issues and were compared with a similar, 
albeit larger, survey conducted by the Department of 
Health. Complementary to this fieldwork, a review of 
current and proposed clinical policies, guidelines and 
legislation and of media commentary was undertaken. 
The study revealed much public confusion and ignorance 
about how people can influence the medical treatment 
they receive when seriously or terminally ill. The study 
revealed an urgent need for Government initiatives to 
provide more information on patient rights in relation to 
consent forms and advance statements. 
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Methodology 


Part of the study involved the examination of 
information sources related to clinical policies 
and guidelines, legislation and media 
commentary and an examination of practical 
strategies for the dissemination of information 
and the implementation of policy. The results 
of this are presented as an extended literature 
review. 

A small-scale survey (the J Survey) was 
carried out to update results obtained in the 
“Omnibus” survey conducted by the 
Department of Health (DoH) (1996), which 
questioned 1,859 people across the UK using 
door-to-door interviews. The survey covered 
a wide range of health-care issues, of which 
advance directives/living wills formed a small 
component; this was concerned with the 
assessment of public knowledge of the forms 
and their implications. It provides a useful 
basis for comparison in terms of timescale, 
volume, and demographics. Since that time 
there has been progress in the development of 
policies for patient rights within health care 
and a growing awareness of associated issues 
such as advance statements. Hence the 
rationale for the J Survey, which attempted to 
assess the extent to which these policies are 
impacting on the public and on health 
professionals. In addition, the J Survey 
extended the range of coverage to explore 
respondents’ attitudes to, and understanding 
of, patient rights in relation to particular 
health situations (including terminal illness). 

The J Survey was devised and carried out in 
mid-2002; this made use of postal 
questionnaires. There were 81 respondents 
over a broad age range, including people from 
various professional backgrounds. Time, 
geographical and budgetary constraints meant 
that the sample was restricted to those living 
within the catchment area of London. The 
Department of Health study represents a 
truer picture of public knowledge in that it 
surveyed a larger number of individuals across 
England and Wales. These limitations have to 
be accepted in a study of this scope. Despite 
these disadvantages, one can assume that, as 
inhabitants of a major city in an affluent area 
of the country, people would have relatively 
good access to whatever information was 
available. Some questions were based on 
hypothetical scenarios in order to examine 
attitudes to the possible withdrawal or 
withholding of medical treatment in certain 
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circumstances. In addition, evidence was 
collected during interviews with a number of 
health professionals, including doctors, within 
the National Health Service (NHS) and other 
Trusts and with several of the respondents to 
the J Survey. These interviews afforded 
valuable insights into values and attitudes — 
including ethical/moral/religious aspects — 
which informed the expressed personal 
judgements on issues relevant to this study. 


Literature review 


Consent forms 

In April 2002 the Department of Health 
introduced consent forms that required health 
professionals to seek the consent of patients 
before administering certain treatments. 
These forms also allow patients to indicate 
which treatments they wish to refuse. The 
forms anticipate a number of different 
circumstances including situations where the 
individual lacks the capacity to indicate his/ 
her wishes, in which case the health 
professional is advised to act in the best 
interests of the patient. The DoH summary 
guidance document for health professionals 
(DoH, 2001a) advises that “If the patient is 
not offered as much information as they 
reasonably need to make their decision, and 
in a form they can understand, their consent 
may not be valid.” The DoH “Good Practice 
in Consent” Advisory Group produced an 
implementation guide (DoH, 2001b), which 
provides guidance to health-care staff, a 
model consent policy, consent forms and a 
guidance leaflet. It makes a series of detailed 
recommendations that health professionals 
should take into account when seeking 
consent. The Group also published additional 
guidance on consent covering the particular 
needs of children, older people and people 
with learning disabilities. These booklets are 
publicly available from their Web site or the 
NHS response line. The Royal College of 
Surgeons also recognises the rights of patients 
to accept or refuse specific treatments in 
section IV of their guidance paper “The 
surgeon’s duty of care” (Royal College of 
Surgeons, 1997). 

NHS Trusts are increasingly studying the 
law and risk management issues and 
developing local guidelines for consent 
(Rashid, 2000). One aspect of this is a greater 
emphasis on information provision. The 
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guidance contained in the information 
booklets aims to ensure that patients are given 
the necessary information and opportunity for 
discussion in order to make an “informed 
choice”. This is critical, since the courts can 
question the validity of an advance statement 
if this is disputed. 

A study by Stapleton et al. (2002), which 
examined the use of evidence-based leaflets to 
promote informed choice in maternity care, 
found that cultural factors influenced the way 
in which information was disseminated 
and inhibited the achievement of a genuinely 
informed choice for patients. The evidence 
provided by Coulter and Dunn (2002) on the 
role of health professionals in the completion 
of consent forms by the patient indicates that 
sometimes there may be an element of 
coercion in gaining consent, and in failing to 
give adequate opportunity for discussion. 

In circumstances where patients lack the 
capacity to make informed decisions at the 
time of treatment the situation is more 
complicated, as instanced in cases considered 
by Baldwin and Hurley (2002). The law 
distinguishes between those who have made 
their views clear when they had the capacity 
and those who did not. If people have not set 
out their views in an advance directive, the 
law provides that they must be treated in their 
best interests. Best interests are understood to 
incorporate the patient’s wishes and feelings 
as well as clinical factors (BMA, 2001). 

A total of 40 per cent of people in the DoH 
survey thought that, if they orally informed 
their relatives of their wishes in advance, this 
would result in their wishes being carried out. 
However, although the involvement of 
relatives/close friends in completing a consent 
form with a patient is encouraged, there is no 
legal basis for this belief: 


One of the most enduring myths about consent 
has been that relatives are empowered to give 
consent on behalf of incapacitated adults. There 
exists, however, no legal basis for this (King, in 
Nys (1991)). 


Even when patients have made explicit their 
wishes in advance (in an advance statement, 
for example), the lack of a statutory 
requirement means there is a degree of 
uncertainty regarding the outcome, especially 
where medical staff are opposed on principle. 
This is of great concern to those who argue 
the case for patient autonomy. It has been 
argued that legal recognition should be given 
to a third party to ensure that medical 
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professionals comply with the previously 
expressed wishes of the patients. 


Advance statements (advance directives/ 
living wills) 

The emergence of advance directives/living 
wills is closely related to the increasing 
complexity of decisions regarding medical 
treatment at the end of life. 


Legal issues 

It is important to discuss the legal background 
to advance statements, since court decisions 
help to direct health policy in the form of 
initiatives and clinical guidelines. A number 
of cases, quoted by Docker (2000), 
established legal precedents for the 
withdrawal of treatment and for living wills. 
One of the most prominent was the 1993 
Tony Bland case[1]. Bland was in a persistent 
vegetative state (PVS) following the collapse 
of the Hillsborough Football Stadium. His 
parents and the hospital authority concerned 
had sought permission from the High Court 
to withdraw the artificial nutrition and 
hydration keeping him alive. He had not 
made anything that could be construed as an 
advance ‘statement[1]. The Appeal Court 
concluded that, if Bland had made an 
advance statement covering non-treatment in 
the event of PVS, the doctors could not only 
lawfully comply but would be acting 
unlawfully if they did not. Lord Butler-Sloss 
said, “all counsel agreed that the right to 
reject treatment extends to deciding not to 
accept treatment in the future by way of an 
advance directive or ‘living will”. However, 
an opposing view was expressed in another 
case considered by Docker (2000), in which 
an advance directive refusing treatment had 
been signed: 


Although a carefully worded living will may be 
decisive evidence in court, the choice must be 
clearly established and applicable in the 
circumstances ... Unless and until that is 
established in any particular case, living wills are 
persuasive and informative documents but 
lacking the full weight of the law at the time that 
they are needed (Docker, 2000). 


In many circumstances patients will be forced 
to seek redress from the courts, relying on 
previous judgments. Even when the law is 
applicable to a person’s situation, it may still 
have to be enforced through lengthy court 
proceedings. There is an obvious tension 
between the law, patients’ rights, and medical 
ethics that can lead to legal disputes as to the 
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validity of an advance statement. The Law 
Commission in 1999 reviewed the need to 
introduce statutory requirements for advance 
statements. They rejected the need to legislate 
— their view was that “the guidance contained 
in case law together with the BMA’s code of 
practice provides sufficient clarity and 
flexibility” (Lord Chancellor’s Department, 
1999a, b). Despite reports of a Government 
initiative to strengthen the legality of living 
wills the situation remains the same. 

Some doctors may also be opposed to a law 
that makes them vulnerable to litigation 
charges. There was unanimous opposition to 
a change in the law from all the doctors who 
took part in the J Survey. 

However, it seems that it is only by 
statutory requirement that there would be any 
impetus for change. It would bring this issue 
into the public domain and place an 
obligation on health professionals to ensure 
that individuals are adequately informed of 
their rights, with the assurance that they will 
be respected. Doctors would need to be 
assured that appropriate legal safeguards 
concerning litigation would be defined. 


Clinical guidelines for advance 
statements 

Advance statements are recognised as being 
legally enforceable by the British Medical 
Association, the Royal College of Nursing, 
the General Medical Council, the Nursing 
and Midwifery Council and the Law Society. 
The policies of the major organisations 
involved are considered below. 


Brittsh Medical Association (BMA) 

Doctors are obliged by common law and 
ethical codes of practice to respect the wishes 
of patients who refuse treatment, including 
advance directives requesting that their lives 
are not prolonged by medical treatment, 
provided that the patient is competent, i.e. of 
sound mind, when making that choice. The 
BMA guidelines emphasise the need to set 
legally enforceable limits and strictly defined 
circumstances in which it would be ethically 
acceptable to withdraw treatment. They also 
recommend that health professionals examine 
their own values to determine whether there is 
a potential conflict in treating a patient whose 
values may be contrary to their own; patients 
must be informed of their doctor’s own views 
and any objections they may have to the 
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principle of an advance statement (BMA. 
1995a). 


Royal College of Surgeons 

Reference has been made to the guidance 
paper “The surgeon’s duty of care” (Royal 
College of Surgeons, 1997). It recommends 
that patients’ rights to refuse treatment be 
respected as long as “the treatments that are 
rejected are clearly specified along with the 
clinical conditions under which they may be 
offered”. 


Royal College of Nursing 

The Royal College of Nursing advises nurses 
against signing or drawing up a living will for 
a patient in case it affects its validity but 
recommend that they encourage patients to 
use one of the standard living wills available. 


NHS Trusts 

Little information is available from NHS 

Trusts about policies and guidelines for 

advance directives/living wills, as distinct 

from consent forms. In a recent study, 

Diggory and Judd (2000) conducted a 

questionnaire survey of 247 NHS Trusts of 

various types to enquire about their policies 
regarding advance directives. Some of the 

findings are as follows (see Table I): 

* Only 30 per cent of the Trusts which 
responded had developed policies, and 
these policies varied widely in their 
approach, content and understanding of 
the legal issues. Many Trusts were aware 
of the problems and 187 Trusts (76 per 
cent) believed that national guidelines on 
advance directives would be useful in 
resolving these. 

*  Djisturbingly, of the 70 Trusts not 
intending to develop policies at that time, 
43 provided acute services, where a policy 
of advance directives is likely to be 
appropriate. However, 63 of these 70 
Trusts (90 per cent) were among those 
that expressed interest in the 
development of national guidelines. 


An increase in general information on these 
issues since 2000 appears to have resulted in 
some improvement in policy development. 
Booth (2003) has recently reported a 


Table | Policies on advance directives within NHS Trusts 


Intending to Not intending to 
develop policy develop policy Other Trusts 


87 70 53 
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significant development at Hammersmith 

hospital: 
[T]he Hammersmith Hospitals NHS Trust in 
West London will be the first in Britain to ask 
patients to write a “living will”. These written 
wishes will inform future treatment decisions 
and override any opinions expressed by the 
patient’s family. The trust decided to introduce 
the form after its study of 74 healthy elderly 
patients showed that, if they were dying, 94 per 
cent would prefer to refuse surgery, 93 per cent 
did not want artificial feeding, 92 per cent did 
not want to be placed on a ventilator and 90 per 
cent did not want to be resuscitated. Women 
were twice as likely as men not to want 
treatment. The Hammersmith Trust intends to 
hand out the forms to elderly patients who are 
about to leave hospital, so that they can take 
them home and discuss them with their families. 
Trained nurses will explain the implications of 
their choices (Booth, 2003). 


Provision of information on advance 
statements 

Groups representing patient rights 

The NHS patient advisory and liaison services 
aim to provide a support service for patients 
and their families. The two main independent 
patient bodies are The Patients’ Association 
and Patient Concern. Other groups providing 
information include Age Concern, the 
Voluntary Euthanasia Society (VES) and the 
Terrence Higgins Trust (THT). 


Electronic resources 

Official sources include NHS Direct, the 
DoH Web site, and the British Medical 
Association. The BMA seems to be the most 
accessible in terms of locating information 
easily. Although the content is geared towards 
practitioners, much of the literature is 
designed to have a practical application; 
therefore the information and advice given are 
clear and precise. There are numerous 
Internet sites but many are based in the USA 
(where living wills are widely accepted) and 
therefore not very helpful because of the 
differing legal position in the UK[2]. The 
Patient SelfDetermination Act in the USA is 
a law requiring that all states recognise living 
wills and Durable Power of Health Attorney 
as advance directives, and that all funded 
institutions provide patient information 
documents, health-care provider training and 
community and patient education 
programmes that discuss advance directives. 
The BBC Web site[3] provides a sense of 
historical context and includes debate on 
some of the moral and ethical issues involved. 
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It also gives access to a comprehensive and 
authoritative range of articles. The VES[4] 
and the THT[5] provide a great deal of 
information on their Web sites, including 
model forms and useful legal and practical 
advice regarding their completion. 


Format for advance statements 

The BMA Advance Statements Code of Practice 
(BMA, 1995b) describes the different formats 
such a document may take. It can be a 
witnessed statement, a written document, a 
signed printed card, or a note of discussion 
recorded in a patient’s notes. Clear general 
statements or preferences should be respected 
if appropriate but are not legally binding. 

Docker (2000) rejected the suggestion that 
the term “living will” be replaced because of 
potential confusion with a “last will and 
testament”. He concluded that “living wills 
are widely discussed in the popular press and 
the term is now known to most health-care 
workers. If it does not cause confusion to the 
man in the street, there is no reason why it 
should do so to lawyers or lawmakers.” 

The term “living will” is often regarded as 
preferable to “advance directive”, because it 
conveys the solemnity of the document and 
gives it a legal flavour. However, it is also 
potentially misleading. In both the DoH 
survey and J Survey, some people understood 
it to mean a financial will. The legal allusion is 
also unhelpful in that people may be 
encouraged to lodge their living will with their 
solicitor or place it in a legal deposit box. 
Indeed, several people suggested that they 
imagined this would be the case. One of the 
main recommendations to the House of 
Lords by Docker (1983) for the Voluntary 
Euthanasia Society for Scotland (VESS) was 
the need for a standard model form or 
document that would be universally useful to 
all sectors of the population as well as 
fulfilling the needs of the medical and legal 
profession. 


Practical arrangements 

Case notes 

Although BMA guidelines recommend that 
patients’ views are recorded in their case 
notes, there is potential for mistakes arising as 
a result of poor communication, particularly if 
several specialists are involved. A few simple 
questions added to the hospital admission 
form would alert hospital staff to the existence 
of a directive. 


Volume 55 - Number 5/6 : 2003 - 357-366 


Cards 

The BMA has recommended that patients 
carry a card indicating that an advance 
statement has been made and where it can be 
found. Medical cards issued by EXIT 
(Scotland) and VES (England) provide 
another means of recording one’s wishes 
regarding non-treatment. 


Register of advance statements 

A register for advance statements could be an 
extremely effective method of ensuring that 
patients’ wishes are consistently recorded. A 
successful scheme will require a concerted 
effort to improve public knowledge, possibly 
through a nation-wide advertising scheme. 
Given the pressure on public health 
resources, this may be unrealistic. 


Armbands 

A scheme similar to an existing one for organ 
donation is offered by MedicAlert, a 
non-profit-making charity. They can supply a 
bracelet or necklace that alerts medical staff to 
the existence of a living will and gives details 
of a contact telephone number. 


Electronic health records 

The importance of good information, 
accessible when and where it is needed, is a 
central theme of the NHS Plan “Information 
for health: building the information core” 
(DoH, 1998). The creation of electronic 
health records is at the heart of this strategy. 
The perceived benefits of this are that NHS 
professionals will have 24-hour secure access 
to critical information relating to patient care. 
The Plan addresses the need for timely and 
accessible information in emergency care, 
where rapid decisions have to be made. 
Doctors and nurses are advised in emergency 
situations to try and establish as far as possible 
whether a living will/advance directive exists 
but clearly in an emergency situation the 
priority will be perceived as saving life. The 
Government has recently proposed the 
centralisation of electronic health records for 
country-wide access. This would be 
particularly useful in emergencies. However, 
there are serious security concerns about 
unauthorised access. In a later NHS Plan 
(DoH, 2000) the Government also 
announced its intention to give all patients 
access to their electronic health records by 
2004: “Electronic access has the potential to 
significantly improve communication and 
accuracy of records” (Coulter and Dunn, 
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2002). This would have clear benefits in 
respect of living wills and advance directives. 
However, security then becomes more 
problematical. 


Dissemination of information 
Goss (2002), of Patient Concern, stressed the 
point that: 


... much of the Government’s efforts to improve 
public knowledge with regard to patient rights, 
specifically living wills and consent forms, has 
concentrated on the design and publication of 
leaflets rather than ensuring this information 
actually reaches individuals concerned. 


There are several factors that will affect this: 
the physical location of leaflets, efforts by 
health professionals to make people aware of 
their existence and opportunities for further 
discussion. Because the criteria for living wills 
need to be exact, they involve a thorough 
awareness of risks, circumstances and legal 
position; ambiguities are likely to be settled 
only in the courts. The experience of the 
USA, which has had a longer experience of 
dealing with such forms, suggests that doctors 
require relevant information and training in 
communication skills in order to advise their 
patients well before admission to hospital. 


Survey results 


The questionnaires and interviews elicited 
many points of interest; these are summarised 
below. 

Many of those surveyed in the J Survey had 
not heard of the terms consent forms and 
advance statements (advance directives/living 
wills); the results are: 

* Those that correctly identified the terms 
as being related to medical care (79, 68 
per cent in the J Survey, 438, 40 per cent 
in the DoH survey) had only limited 
knowledge of what was involved. Even 
allowing for the fact that some people 
chose the option to instruct both a doctor 
and a relative about their wishes for 
medical care, a fairly large proportion in 
the DoH survey (443, 40 per cent) were 
under the mistaken impression that they 
need only instruct their relatives. 

* A total of 79 (63 per cent) had heard of 
the term living will; this compares with 
992 (38 per cent) in the DoH survey, 
though differences in the numbers and 
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professional status of those surveyed must 
be considered here and ‘elsewhere[6]. 

* Fewer people were familiar with the term 
advance directive (79, 17 per cent, J 
Survey), though this was higher than that 
found in the DoH Survey (380, 5 per 
cent). 

* A total of 328 (62 per cent) of people in 
the DoH Survey were unaware that they 
could influence in advance the medical 
care they receive. 

* A total of 72 (12.5 per cent) of 
respondents thought that a living will 
dealt with financial arrangements. 

* Less than 40 per cent (79) recognised 
that there had to be written evidence of a 
living will for treatment to be legally 
withdrawn. This was backed up by 
anecdotal evidence from nurses, many of 
whom remarked that relatives were often 
distressed to discover that this was 
needed. 


Female respondents had generally higher 
knowledge and understanding of many 
aspects of the relevant issues studied than 
males. This is ascribed to their often greater 
role in healthcare, both professionally and 
privately. 

An NHS nurse interviewed for this study (J 
Survey) commented that, in her varied 
experience, patients within physical care units 
were given little opportunity to discuss their 
treatment and there was often pressure on 
patients to comply. Where disagreements 
arose, “it depended on how articulate the 
patients expressed their wishes as to whether 
their views were taken seriously”. If patients 
were adamant, usually the medical team 
would endeavour to explore alternative 
treatments. She commented that within 
mental care there was greater emphasis on 
patient involvement in discussion of 
treatments. Because they were dealing with 
the patients’ overall psychological wellbeing, a 
more holistic approach was taken. 

The survey indicated a growth in awareness 
and knowledge of the issues involved 
(particularly of the patient’s right to refuse 
treatment), since the DoH survey in 1996 — 
though many misconceptions remain. A total 
of 67 per cent of those aware of living wills 
had gained their knowledge from the media. 
In some cases the term was associated 
(wrongly) with euthanasia. 
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The results of the J Survey indicated: 
support for the larger-scale DoH survey 
findings that lower age groups showed more 
awareness of the term “living will” than the 
upper age group. The DoH results are: 

* 25-44 year age group: 375, 42 per cent; 
and 

* 65 years and over age group: 244, 24 per 
cent. 


This is somewhat surprising in that one might 
expect that the older group of 65 years and 
above, the group most likely to find these 
matters most relevant to their situation, 
would show greater awareness. Media 
influences may be responsible. Education 
(e.g. in GCSE courses) is an additional factor 
contributing to awareness in the lower age 
groups. 

Nearly 80 per cent of those questioned felt 
that information on advance directives/living 
wills should be made available to all adults, 
regardless of their age or state of health; 
nearly 70 per cent stated that they would 
consider writing one in the future. There were 
indications that religious beliefs were a major 
factor for those who disagreed. Many 
respondents stated that their decision to 
consider writing an advance statement would 
depend on the complexity; the process was 
seen as bureaucratic (i.e. it would involve 
approaching several different people or 
organisations). Most people (69, 74 per cent) 
felt that it would be easier if all information 
and forms were available from a single source, 
preferably GP surgeries. However, a 
significant number of people (69, 26 per cent) 
felt that information should be publicly 
available from a number of sources, e.g. post- 
offices, libraries. Although many people 
surveyed in this study felt that leaflets were an 
ideal way of increasing awareness, this had to 
be backed up by a support system, either 
through their general practitioner (GP) or 
through an independent consultant, to offer 
advice and opportunities for discussion. 

All but one of the eight GPs were in favour 
of living wills. Opinion was divided as to 
whether information should be made 
available to all adults, with four doctors : 
expressing agreement, the remainder being 
uncertain. Leaflets within surgeries were 
unanimously supported. Three out of the 
eight doctors interviewed indicated that their 
practice had clearly established guidelines. 
There was some uncertainty amongst the 
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remainder. All except one felt that the current 
arrangements were unsatisfactory. One of the 
main concerns expressed in the 
questionnaires from doctors was the potential 
uncertainty that arises where a significant 
time has elapsed between drafting the will and 
its enactment. The other frequent concern 
was that people often make inaccurate 
predictions about the quality of life that they 
would be willing to accept after undergoing 
major surgery. A lawyer interviewed in this 
study said that one of the weaknesses in the 
current system is that in emergency situations 
there is no time to establish the existence of ¢ 
living will. Armbands would be valuable in 
such circumstances. 

Some questions required respondents to 
consider hypothetical scenarios featuring 
medical circumstances; in each case they were 
asked if they felt it would be acceptable to 
withdraw/withhold treatment. The results 
from these are interesting. Responses to such 
questions that attempted to assess people’s 
views with regard to advance statements 
showed significant support for these 
statements, particularly for patients suffering 
extreme pain or with a terminal condition. In 
scenarios where the patient had suffered a 
stroke, was not in pain, but where quality of 
life was greatly affected, 73 (50 per cent) of 
respondents approved of assistance to die 
being given by medical staff (e.g. by 
withdrawal of treatment) on the basis of either 
a patient’s oral request or a previously 
prepared living will. The number of 
respondents who disapproved of such action 
being taken in these circumstances (73, 24 
per cent) was decreased to only 12 per cent if 
the patient had made a living will — the other 
12 per cent converted to “Not sure”, possibly 
because of lack of knowledge of living wills. In 
terminal cases where there was a living will, a 
high proportion (80 per cent) approved of 
termination (e.g. by withdrawal of treatment); 
this was increased to 74 (90 per cent) where 
there was extreme pain and a patient 
requested (or had previously requested) 
action for termination. 


Conclusions 


The study indicates that people are often 
confused about the terms classified as 
advance statements (consent forms, living 
wills, etc.), of how they function, and of their 
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legal form status for effective action. 
Moreover, everyone interviewed said they did 
not know how to obtain information about 
them and were unaware of the practicalities of 
completing an advance statement. In 
particular, many were unsure about who 
should be informed. 

The results provide a disturbing 
confirmation of the public ignorance about 
these matters, found three years ago, in a 
study by Schiff et al. (2000), though there is at 
least a growing awareness of, particularly in 
the younger age groups, and strong support 
for their availability and practical use. Patients 
now expect to have greater autonomy in 
health care, although they are unsure both of 
how to obtain it and of the extent of their 
“rights”. In particular, the introduction of 
consent forms within the NHS is a major 
advance in the context of ongoing medical 
treatment. However, this study supports the 
view that more effort is needed to ensure that 
practical measures to provide information on 
consent forms to potential patients, with 
back-up advice, are defined and 
implemented. The consent form makes a brief 
suggestion that it may be appropriate to make 
use of independent advocates, i.e. health-care 
proxies. The role of such health-care proxies 
could be to provide information to patients 
and explore possibilities, which could then be 
discussed in more detail with their doctor 
when making decisions. It would ensure that 
the principles outlined in the guidelines are 
consistently applied by professionally trained 
advocates, who can give consistent and 
objective advice free from the personal 
feelings which may intrude as a result of being 
solely involved in their care. It would be very 
useful to have a similar arrangement for living 
wills. 

The subject of advance directives/living 
wills poses more difficult problems. This 
study found that the understandable reticence 
to discuss issues of mortality is likely to be one 
of the main reasons why there is little 
knowledge or understanding of the subject. 
What is lacking is any official initiative to 
inform the public of their rights with regard to 
advance directives/living wills. This role is 
performed almost solely by the media through 
reporting of legal cases. This has possibly had 
the effect of improving awareness of terms, as 
indicated by the J Survey, but has not 
improved understanding for the majority of 
people. 
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The inevitable effect of this lack of 
knowledge appears to be a presumption 
amongst some health professionals that 
people are indifferent. It is certainly implied 
by the relative lack of information within the 
health service. 

The recent survey of NHS Trusts by 
Diggory and Judd (2000) revealed a 
disturbing lack of policies on living wills/ 
advance directives. The policies examined 
showed a wide variation in approach, content 
and consistency in understanding of the legal 
issues. However, there was a high level of 
interest in national guidelines being 
developed. Clearly, national guidelines would 
aid the promotion and development of 
policies within Trusts and also of consistent 
approaches to the issues involved. They 
would also be very important in informing the 
wider public and the media of the legal 
provisions that are available. 

There is a case to be made for government 
to take a more active approach to 
disseminating information, perhaps as part of 
patient care through surgeries. The lack of a 
standard form for a living will available 
through the NHS affects the accessibility 
of these agreements and is likely to influence 
their acceptance. A common response to the 
living will from interviewees to the model 
forms by the THT and VES was to question 
whether this was a “real” form, i.e. would it 
have any effect in an end-of-life situation. 
Health professionals may have insufficient 
time to thoroughly discuss these matters with 
patients. Any appropriate safeguards such as 
counselling and discussion will require greater 
resources of time and money. The suggestion 
by Docker (1983) that the paper work for 
living wills should include explanations as 
well as safeguards would benefit both patient 
and doctor. Doctors quite rightly respond to 
different patients in different ways, and this 
depends in part on their perception of a 
patient’s ability to cope with such 
information. For this reason, it may be 
inappropriate for doctors to raise the issue 
with some of their patients. Leaflets within 
surgeries would be valuable but may well be 
inadequate for some. 

The BMA recommends that people should 
lodge a copy of their advance directive/living 
will with their doctor and that a relative 
should be informed. For legal reasons it is 
imperative that it is lodged in their doctor’s 
notes and is witnessed. Given that this 
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document provides the most reliable source of 
evidence in a court of law, there need to be 
great clarity and a more systematic procedure 
for lodging people’s directives. The 
Government initiative to introduce electronic 
records would help overcome some of the 
practical difficulties, particularly in 
emergency situations. NHS trusts are taking 
this issue seriously, as evidenced by the 
reference to advance statements in guidelines 
and policies for consent. 


Recommendations 


Much more effort needs to be put into making 
information about consent forms and advance 
statements available to patients and to the 
general public and into advising patients of 
their use and implications well before they 
enter hospital. More vigorous government 
initiatives on these matters and on developing 
national policy guidelines are required. GPs’ 
surgeries are perceived to be an important 
potential source of information. 

Much of the development of advance 
directives/living wills has been undertaken by 
organisations such as the THT and the VES, 
and by similar organisations in other 
countries. In some countries such documents 
have a firm legal basis. Based on the 
evaluation in this study of the current legal 
situation and on the associated survey of 
people’s views, there appears to be a need for 
national incentives to review the various types 
of advance directive/living will with a view to 
formulating a standard UK version acceptable 
to the medical establishment and patients and 
national guidelines for its use. The aims 


should include: 
* to work towards putting such documents 


on a firm and clear legal footing; 

* to promulgate information about 
procedures to be followed by doctors, 
other medical staff and patients in 
completing and lodging such a document 
and using it as a basis for appropriate 
action; 

* to actively promote training in such 
procedures for relevant medical staff and 
independent advisers. 
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Notes 


— 


Airedale NHS Trust v. Bland [1993], WLR 316 [1993] 
1 All ER 821. 

2 www.uslivingwill registry.com/internet_ 
resources.shtm; www.mindspring.com/~scottr/ 
will.html; www.nwjustice.org/docs/9501.html 
www.bbc.co.uk 

www.ves/org.uk/livingwills. html 
/www.tht.org.uk/publications/livingwill.htm 
Differences may be accounted for by demographical 
variations as well as size of the sample. The sample 
group for the J Survey constituted only 10 per cent 
in size of the DOH survey, most of the respondents 
being drawn from the London/SE region and 
consisting of a large number of professionals. 
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The design and development of a computer-aided learning 
toal for the education of patients with diabetes and for the 
training and support of practice nurses providing diabetes 
care have been previously reported. diabCAL is a broad and 
comprehensive learning tool, which can be accessed as a 
modular course or a desktop quick-reference. Evaluation is 
central to the design process and can be formative, 
informing design and development throughout the lifecycle, 
or summative, assessing the value of the finished product. 
diabCAL was subject to rigorous evaluation at various 
stages during development, including: verification of 
curriculum content and structure, evaluation of navigation 
and evaluation of system content. A final hybrid formative- 
summative evaluation assessed usability and acceptability 
of the system. Evaluations were conducted by technical 
experts and users, and findings of the final evaluation study, 
conducted by patients, confirmed that diabCAL was 
acceptable to most users — that is, interesting and perceived 
to be useful — even to those with little previous computer 
experience. 
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Introduction 


Computer-aided learning (CAL) has bean 
established in recent years as a useful tcol 
(Lowe et al., 2001; Devitt et al., 2001; 
Campaign for Learning, 2000; Bauer and 
Huynh, 1998), and is increasingly employed 
in patient education (Graham er al., 2000; 
Campbell et al., 1999; Day et al., 1997; 
Carroll et al., 1996; Chambers and Frisby, 
1995). Diabetes is a non-communicable 
chronic disease affecting increasing numbers 
of people, which in the UK and world-wide is 
a matter of growing concern. It is the focus of 
increasing attention, and was the subject of 
one of the first National Service Frameworks 
published by the UK Department of Health 
(DoH, 2001, 2003). Diabetes is a complex 
condition, the (self-) management of which 
requires significant knowledge and 
understanding, and (self-) monitoring of the 
condition requires both knowledge and 
technical skill. Patient education is therefore 
seen as the corner-stone of diabetes care. As 
the prevalence of diabetes increases, 
responsibility for the care of people with 
diabetes increasingly falls to primary care, and 
often to practice nurses; it is therefore 
essential that they are equipped with the 
knowledge, skills and support to care for 
increasing numbers of patients. 

In order to address the problem of 
educating a large — and growing — population 
of diabetic patients about effective self-care, 
within the time and cost constraints of the 
NHS, a dual-branched, computer-aided 
learning system was developed. diabCAL was 
developed with specific branches aimed at 
patients, on the one hand, and practice 
nurses, on the other. It constitutes a novel 
intervention to improve patient care ard self- 
care in two ways: by educating patients 
themselves and by training the practice nurses 
increasingly responsible for their care; in 
effect, allowing the NHS to “treat more 
patients and provide the highest quality of 
care” (Cook, 2000). The design and 
development of this computer-aided learning 
tool have been previously reported (Berridge 
et al., 2000). 


Emmaz-Jane Berridge was supported with z CASE 
studentship from the Economic and Social 
Research Council. Ethical approval for the 
usability evaluation involving patients was gained 
from St Thomas’s Hospital Local Research Ethics 
Committee. 
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Evaluation is central to the design of 
computer-aided learning, and can be 
formative or summative. Formative 
evaluation is conducted throughout the 
lifecycle, informing design and development; 
summative evaluation assesses the value of the 
finished product (Sutcliffe, 1995). diabCAL 
was subject to rigorous evaluation at various 
stages during development, providing expert 
and user feedback to enhance the final 
product. A final hybrid formative-summative 
evaluation was conducted to ascertain the 
usability and acceptability of prototypical 
modules and system tools to real world users. 
This paper summarises the various 
evaluations conducted and their findings. 


Verification of curriculum structure and 
content 


Early in the design process, a brainstorming 
exercise and domain analysis were conducted 
to provide an understanding of the broad and 
complex domain of diabetes. From this 
process, concept maps were developed, which 
both mapped the domain and provided a 
curriculum for the learning tool. Having 
determined that diabCAL would be modular 
in structure, individual concept maps were 
developed for each module, and higher level 
maps were drawn to represent the top-level 
structure of the system. In order to verify that 
the information contained was correct and the 
structure sensible, these concept maps were 
subject to evaluation by domain experts. 
During their development, verification of 
structure and content was obtained through 
consultation with members of the 
multidisciplinary team at St Thomas’s 
Hospital, London, whose advice was sought 
at regular intervals. Specialist nurses, a 
dietitian and podiatrist all contributed 
informal feedback, which was taken into 
account as development of the maps 
progressed. In order to assess the overall 
organisation and content of the concept 
maps, upon completion they were subject to 
formal evaluation by another group of domain 
experts. This evaluation sought to verify the 
structure and content of the curriculum, as 
embodied in the collection of concept maps. 
The maps were sent, along with a 
straightforward questionnaire, to a sample of 
Diabetes Specialist Nurses (DSNs) and other 
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specialist professionals involved in diabetes 
care from around the UK. 

The questionnaire requested scored 
responses to the perceived completeness of 
the information contained in each concept 
map, and therefore in each module and in the 
higher level structure of the proposed system. 
These scores were on a scale from 1-6 — 
“incomplete” to “complete”. The aim of the 
verification exercise was to establish to what 
degree domain experts agreed with the 
structure and content as proposed within the 
concept maps, as well as to elicit any further 
information deemed necessary for inclusion in 
an educational system for patients and non- 
specialist nurses. The ideal score for each map 
was therefore 6, indicating complete 
agreement with the map in terms of both 
structure and content. Where the score for a 
map fell below 6, the questionnaire allowed 
space for free text responses to the 
organisation, content and structure of the 
information. Where there was disagreement 
with content and structure, comments were 
taken into account and the concept maps 
modified accordingly. 

Evaluators were initially approached by 
telephone or by letter. An individual covering 
letter was sent with the questionnaire and 
concept maps to each evaluator, explaining 
the aims of the project and the purpose of the 
evaluation. 


Findings 
Findings from this evaluation have been 
reported elsewhere (Berridge er al., 2000). In 
summary, of the 14 modules in the patient 
branch of the system, 29 per cent achieved a 
mean score of 5.5 and above. This high 
degree of consensus indicated that few 
changes were to be made to an individual 
module’s content and organisation. A total of 
64 per cent achieved a good degree of 
consensus — a mean score of 5 and above — 
and all maps attained a mean score above 4.5. 
These results indicate that there is 
considerable agreement by health-care 
professionals regarding the completeness of 
the maps. Of the nine practice nurse maps, 44 
per cent achieved a mean score of 5.5 and 
above, 89 per cent a score of 5 and over, and, 
again, all achieved a mean score of 4.5 and 
above. 

Only two modules of the total of 23 
obtained scores below 4. The score of 3 
indicates that a substantial amount of work 
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was necessary on structure and content of the 
module. It should be noted that these scores 
were decided after significant discussion of 
the maps between evaluator and facilitator, 
which was an unusual situation — each of the 
other evaluators had considered the maps and 
completed the questionnaire independently. 
All scores given by this evaluator tended to be 
lower than those given by others. It is 
suggested that any opportunity for discussion 
generates ideas and considerations that may 
not come to light under independent scrutiny, 
and that this may be a reason for the lower 
scores. 

Each concept map was subject to 
consideration as a discrete entity, and yielded 
a discrete score, along with comments as to 
what was thought to be lacking, where 
appropriate. All evaluators’ scores for each 
map were taken together to show an average 
score and to provide the “percentage of the 
ideal”, showing to what extent the modules’ 
organisation and structure deviated from 
what the polled group of experts perceived to 
be “ideal”. Evaluators’ comments for each 
module were also considered together to 
provide a picture of all the modifications 
deemed necessary by these experts in the 
creation of comprehensive and well 
structured modules. The concept maps were 
duly modified according to the domain 
experts’ comments, many of which requested 
only trivial changes, or flagged the importance 
of including more detailed information than 
the high-level maps presented. 

Having verified and modified the concept 
maps thus, development of diabCAL 
continued, and modules began to take shape 
from this skeleton built of consensus. 


Navigation evaluation 


The second exercise in formative evaluation 
involved the assessment of the navigation 
procedure, which was necessarily developed 
in the early stages of content creation. 
Navigation was broadly based upon 
Windows© conventions. Because the two are 
inextricably linked, navigation was designed 
as the interface was designed and 
implemented. 

Evaluation of the navigation methodology 
was undertaken by a sample of 17 practice 
nurses. Participants were visited at their 
workplace by the investigator, with diabCAL 
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installed on a laptop PC. Each evaluator was 
asked to undertake a short set of tasks, in 
which they were required to navigate from 
one specific point in the system to another; 
having performed these tasks, they were asked 
to complete a short questionnaire about their 
experiences. 

A set of four tasks was developed. Three of 
these involved participants navigating through 
parts of the system from one specific point to 
another, while the fourth requested that 
participants seek out information of their own 
choice within a specified module sub-topic, 
which they had to reach themselves. The tasks 
were designed with two objectives in mind. 
First, tasks were designed to require the use of 
both the module menu and sub-topic list (see 
Figure 1), as well as other interaction 
techniques such as buttons and hypertext 
links, therefore forcing participants to fully 
exploit all the available navigation techniques 
(rather than rely on familiar or obvious 
techniques, such as the clicking of buttons). 
Second, the tasks were designed to take users 
deep into the structure of the system in order 
to ascertain whether they felt comfortable and 
well-oriented even at a relatively deep level. 

While the tasks were designed to force 
participants to use all available navigation 
tools and to venture deep into the structure of 
the system, the questionnaire endeavoured to 
reveal — based on participants’ experience 
with navigating through the system in this way 
— whether the navigation method employed 
was: 

e Meaningful: does navigating in this way 
make sense? 

*  IJntuitive: having read the instructions, can 
users navigate in this way? 

: Straightforward: is it easy to navigate in 
this way? 


In addition, the questionnaire sought to find 
out whether users: initially read the navigation 
instructions provided upon entering the 
system, whether these instructions were 
sufficient and sufficiently accessible, and 
whether the organisation of the system into 
modules and sub-topics — which helps dictate 
how navigation can proceed — was 
meaningful. 

The questionnaire was divided into sections 
about: 
* Navigating as a new user. 
- The navigation instructions (introduction 


page). 
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Figure 1 The diabCAL user interface, with the module menu at the top of the window and the sub-topic list (with 


current location highlighted) on the left 














File Module Menu Tools 


module 
{Practical 8 Skills ? 


This sub-topic shows you in words and video demonstrations how and 
E 5 aeaaea where to injeci if you are treated with insulin. Clearly if your diabetes is 
treated without insulin, you can ignore this section - the other practical 
skills are important however you're treated, though, so dont ignore 


sub-topic 


te Urine Testing — 
Footcare 

+ Practical Skills FAQs 
t Practical Skills Quiz 
4 


those! 


Select from the list below to begin: 


How do } inject? 

Where should | inject? 

Should | always use the same site? 
When should | inject? 


If you use a syringe - types of syringe, drawing up insulin etc. 





* The module — sub-topic organisation. 
* The control panel. 
* Computer access and confidence. 


The questionnaire was designed with 
guidance from Oppenheim (1992) and 
Bowling (1997). It comprised mostly closed 
questions requiring a yes/no response. In 
some places, these questions were followed up 
with an open question requiring details (the 
“why” or “how” or “where”), or with a list of 
options from which one or more responses 
could be selected. Only one Likert-style scale 
was used, and this was the final question, 
requiring participants to respond on a five- 
point scale about how confident they felt 
using a PC. 

Practice nurses already involved in diabetes 
care were chosen as evaluators at this stage, 
since they represent one section of the target 
audience for the learning tool under 
development and because a cross-section of 
computer literacy was expected. A certain 
amount of diabetes-specific knowledge was 
pre-supposed in order that users could be 
given simple cues about to where to navigate, 
rather than step-by-step instructions on how 


If you use an insulin pen 
How do | dispose of old syringes and pen needles? 








to navigate there; for this reason patients were 
not involved in evaluation at this stage. 


Findings 

Questionnaire results were analysed to find 
participants’ reactions to the system. The 
majority found the system easy to navigate 
(94 per cent), and found themselves well- 
oriented most of the time (94 per cent). 
However, as many as 11 participants (65 per 
cent) still felt that the navigation could be 
improved in terms of ease of use for new 
users. Most popular of the proposed additions 
was the inclusion of a map of the system 
(accessible from the menu bar or from a 
button on the control panel), an option 
selected by 72 per cent (n = 8) of those 
respondents. Next most popular was the 
addition of a Web browser-style “forward” 
button (36 per cent; n = 4), and a further 
three respondents (27 per cent) felt that more 
internal navigation instructions, such as 
“Return to... ”, “Jump to... ”, etc. would be 
helpful. One nurse was particularly emphatic 
about the need for prompts of this kind to aid 
orientation: “I used internal navigation 
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prompts more than I used ‘back’ button etc. I 
feel continuous prompts are essential to ease 
navigation from PC illiterates [sic].” 

Although 94 per cent of users (n = 16) 
found themselves generally well-oriented 
most of the time, 29 per cent (n = 5) admitted 
they did find themselves lost or confused 
about their location on one or more 
occasions. Comments about location 
included “returning to modules”, suggesting 
that the method of choosing modules from a 
menu on the menu bar was not immediately 
obvious for some users. However, despite 
initial problems encountered by some users, 
evaluation confirmed that the method of 
choosing modules from a menu bar item and 
selecting sub-topics from a single select list 
box made sense — this was found to be 
sensible by all users — and that the inclusion of 
internal navigation instructions (hypertext 
links directing users back to the sub-topics’ 
main page) is useful alongside the sub-topic 
single select list box (82 per cent). 

As anticipated, a range of computer use and 
confidence was found among participants, 
though none of the evaluators felt entirely 
lacking in or entirely full of confidence. A 
total of 12 participants (71 per cent) reported 
that they had a PC (running Windows) on 
their desk; however, for one this “depends 
which room I’m working in”. 

The findings illustrated that the _ 
deployment of multiple tools to perform the 
same navigational task could be useful, since 
different users attempt the same task in 
different ways. For example, one participant 
noted that she “got stuck on page [sic] which 
had no backward internal navigation” — that 
is, a page without an explicit hyperlink back to 
the previous page visited. The “back” button 
~ always visible on the control panel — 
performed the same task but, although this 
was mentioned in the instructions and was 
available at all times some new users clearly 
are not aware of it. The additional internal 
navigation instructions (in the form of 
hyperlinks on the page) are a useful addition 
for some users. 

The method of choosing modules from the 
“module” menu at the top of the window was 
not initially obvious for several users, as 
concurrent informal observation of users 
confirmed. However, this method was 
learnable and most users settled into selecting 
modules in this way; indeed, all 17 evaluators 
responded positively in a later question about 
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whether it made sense “to select modules 
from a menu bar item”. 

Evaluation at this stage identified a number 
of modifications expected by users to ease 
navigation. The system was modified 
according to evaluation results and 
participants’ suggestions on how navigation 
could be improved, with priorities based on 
the number of evaluators for whom a 
particular modification was important. 
Several users expressed a wish for the 
navigation instructions page to be accessible 
from the help menu at all times, as well as 
available on entry to the system (88 per cent). 
One nurse “could not find [her] way back to 
it”, and this impacted upon her experience of 
using the system. As a consequence, the 
system was modified to include a “Help” 
button on the control panel — visible at all 
times ~ providing access to navigation 
instructions, as well as a “Help” menu on the 
menu bar — a familiar feature of Windows 
programs — providing duplicate access. Some 
evaluators felt that a map of the system would 
be a useful addition, and so an interactive 
tree-like map was developed, again accessible 
from a button on the control panel as well as 
from an option in the “Tools” menu. Table I 
summarises modifications made on the basis 
of evaluation results. 

With these modifications a new iteration of 
the interface was produced: evaluation 
informed system design and development, 
contributing to the evolution of the system. In 
order to validate the modifications made 
based on users’ comments, and to ensure that 
the navigation procedure continued to satisfy 
the majority of users, navigation issues were 
revisited in a later study. 


Content evaluation 


Having evaluated diabCAL’s navigation 
procedure and implemented the suggested 
modifications, development of the system 
proceeded with the addition of textual and 
multimedia content. While textual content 
was developed with reference to various up- 
to-date medical texts and consensus-based 
sources, it was important that it should be 
validated by experts to ensure the accuracy 
and propriety of the learning material. As 
such, once the textual content of modules was 
implemented, each module was subject to 
evaluation by Diabetes Specialist Nurses. The 
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Table | Summary of modifications to aid navigation for new users, based on evaluation findings (navigation evaluation) 


Modification 








Enhanced accessibility of “How to Navigate” page 88.23 per cent of all participants (n = 16) 
47 per cent (n = 16) (72 per cent of the 11 users indicating that 


Addition of a map of the system 


Number of users for whom this was a priority 


improvements could be made) 


Elaboration of navigation instructions 


evaluation of the first two modules, What is 
diabetes? and Practical Skills, is reported at 
this stage. 

In order to examine the validity of content, 
participants were asked to read sequential 
pages of textual content and score each page 
in terms of the accuracy, appropriateness, 
tone and readability of the text (see Table ID, 
using a six-point Likert scale for each on a 
questionnaire administered by the 
investigator. Each point on the scale was 
discrete, and participants were asked to 
choose one value on each scale, where 1 
represented “not at all (accurate, appropriate 
... etc.)” and 6 represented “completely 
(accurate/appropriate ... etc.)”. A six-point 
scale was chosen over a five-point scale in 
order to deter clustering around the middle 
value. Where evaluators disagreed with 
content in terms of accuracy, they were 
prompted verbally (though prompting was 
often unnecessary) to explain what they 
would change. This information was noted by 
the researcher directly on to the 
questionnaire. Likewise, if any other feature 
of the text (appropriateness, tone, readability) 
was scored poorly, feedback was requested if 
not volunteered and was again noted by the 
researcher. 

Diabetes Specialist Nurses were chosen as 
evaluators at this stage, since expertise in the 
field was absolutely vital to ensure that 
content was appropriately evaluated; from a 
pedagogical perspective, DSNs were chosen 
because of their role as educators. Twenty 
DSNs from across London participated. 


Table II Assessment criteria, content evaluation 





Criterion Definition 

Correctness Is the information conveyed accurate? 

Appropriateness Is content appropriate: to the given audience; to 
diabetes care; within a patient education system? 

Tone is the tone of the text appropriate and balanced: is 
it supportive and unpatronising? 

Readability Is text easy to read: is the readability level 


appropriate; does it make sense? 





29.4 per cent of all users felt the instructions were “not sufficient” (n = 16) 


Findings 


The findings are presented in Tables III and 
IV, which express the frequency of the various 
mean scores achieved for correctness, 
appropriateness, tone and readability of d 
content across the total number of pages of 
textual content in each of the two modules 
evaluated at this stage. 

The great majority of pages were awarded 
high scores (scores of 5 or 6) by DSNs. As a 
result, there was a great clustering of mean 
scores at 5.25 and above. A score of 6 
indicated that the evaluator was “completely” 
happy with content in terms of its accuracy, 
appropriateness, tone or readability and 
would therefore wish to change nothing. A 
mean of 6 therefore indicates consensus in 4 
satisfaction among the group of evaluators 
across the four completed evaluations of the 
module. 

In terms of accuracy, as many as 26 of the 
total of 63 pages (41.27 per cent) in the What 
is Diabetes? module were deemed to be 
completely correct across the four 
evaluations, and this is reflected in the 
frequency of means of 6 in Table III. In the 
Practical Skills module (Table IV) scores 
again tended to cluster around the higher _ 
values (5 and 6) . However, although their 
frequency is low, there were more instances of 
much lower mean scores in the evaluation of 
this module. For example, three pages of 
content achieved mean scores of 3.5 and 
below, for correctness or appropriateness, 
indicating that at least some of the DSNs 
evaluating did not agree with the content 
included. 

That the majority of pages achieved scores 
of 5 and 6 indicates that DSNs were happy 
with the accuracy, appropriateness, tone and 
readability of much of the content. Where 
unhappy, comments were provided as to how “4 
to improve or correct content. The majority of 
contention and variation between scores 
occurred in the Practical Skills module; What 
is Diabetes? dealt more with factual 
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Table III Mean scores for each feature of textual content within the What is Diabetes? module 





Text feature 


Number of pages with a mean score of: 
5.33 5.25 5.0 4.75 4.625 4.5 


6.0 5.75 5.67 5.5 











the evaluation as a whole provided expert 
advice on how to improve content. Having 
already evaluated the curriculum with system 
map evaluation described above, this 


Total 
4.25 3.875 3.25 pages 











Correctness 26 8 1 12 0 5 3 3 1 1 2 1 0 63 
Appropriateness 32 13 1 5 1 7 3 0 0 0 0 0 1 63 
Tone 28 24 0 5 0 6 0 1 0 0 0 0 0 64 
Readability 31 20 0 8 0 3 1 0 0 Q 1 0 0 64 
Table IV Mean scores for each feature of textual content within the Practical Skills module 
Number of pages with a mean score of: Total 
Text feature 6.0 5.75 55 5.25 50 4.75 45 433 3.5 3.25 2.0 Pages 
à Correctness 30 1 7 8 2 1 1 0 1 0 1l 62 
Appropriateness 45 8 3 3 0 0 1 1 0 1 0 62 
Tone 37 22 2 1 0 1 0 0 0 0 0 63 
Readability 37 29 3 3 0 0 0 0 0 0 0 63 
information, which was therefore subject to provided a second verification of the macro 
less dispute. content (the sub-topics and their composite 
As might be anticipated, some markers pages) as well as verification of micro content 
were harsher than others: some DSNs (the detail forming the flesh on that skeleton). 
awarded scores as low as 1 for content with This examination of the micro content 
which they disagreed, while others went only generated unique advice about emphasising 
as low as 3 as their least satisfied. However, all certain terms or phrases, or rearranging 
2 participants were happy to award scores of 6, sections to optimise the impact of what the 
and those who gave much lower scores when DSNs deemed to be key messages. The 
they saw flawed content were no less generous evaluation had the added benefit of providing 
with their sixes. a proof-reading service, picking up 
We do not know if a score of a particular typographical and spelling errors, which 
value meant the same thing to each evaluator despite careful checking had still slipped 
(other than 6 and (probably) 1, for which through. 
explicit meanings were advised), and in some For both modules, evaluation findings 
instances there was substantial variability indicated that the majority of nurses were 
among scores awarded between evaluators. generally satisfied with the content presented 
For example, in the page “Why test urine for in terms of each of its four named features — 
glucose?” within Practical Skills, scores for accuracy, appropriateness, tone, and 
correctness varied from 1 to 5 (individual readability. Comments from participating 
4 scores awarded were 1, 3, 5 and 5). However, DSNs, explaining why they were not 
comments provided (about why content was completely satisfied where scores of less than 
not “completely correct”) varied little 6 were awarded, and suggestions for 
between nurses. In this way, the comments appropriate or requisite modifications were 
provided — the qualitative data collected — taken into account in order to develop the 
were invaluable, while the quantitative next iteration of diabCAL. 
information collected was simply “useful” 
and aided the presentation of evaluation 
findings. Hybrid formative-summative 
The individual scores awarded and the evaluation: usability and user 
mean scores for each feature demonstrated acceptance 
the extent to which there was agreement with 
X content across and between evaluators, and Having subjected various aspects of the 


system to assessment by different groups 
during the course of development, and with 
expert and user feedback incorporated into 
the development process (formative 
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evaluation), finished prototype modules were 
finally subject to evaluation by real-world 
users. Where previous evaluations of 
diabCAL concerned the structure, design and 
content of the system, this final evaluation of 
the two completed modules concerned 
usability (in greater breadth than the 
navigation evaluation) and broader user 
acceptance. 

A summative study can be defined as a 
“study designed primarily to demonstrate the 
value of a mature information resource” 
(Friedman and Wyatt, 1997, p. 304). In the 
context of the current research, this might 
involve a large-scale assessment, requiring a 
longitudinal study of patients’ and practice 
nurses’ learning, conducted to assess the 
usefulness of the finished product from a 
pedagogical point of view — a study of a new 
educational intervention. A study of this scale 
fell outside the scope of this research, in 
which a prototypical system — a representative 
sub-set of the large-scale finished product — 
has been designed and developed. However, 
to assess user satisfaction with the system as it 
stands was within the scope of the project, 
collecting information on the perceived 
usefulness of the system, and various other 
measures of acceptability by users. This was 
really a formative-summative hybrid, acting as 
it did to assess satisfaction with finished 
modules and informing the development of 
subsequent modules. Indeed, Sutcliffe (1995, 
pp. 224-5) offers a different definition of 
summative evaluation: 


... evaluation when the product exists. In this 
case the results of tests add more to the sum of 
knowledge about a product [and], if the results 
can also be used to improve the design, so much 
the better. 


Sutcliffe also expresses that the boundary 
between the two types of evaluation does 
become blurred. 

The evaluation conducied here defines 
usability in terms of some of Nielsen’s 
Usability Attributes (Nielsen, 1993): 
Learnability, Error rate and User Satisfaction; 
and user acceptance in terms of his “social 
acceptability”: do patients enjoy the 
experience of using the system, do they value 
the experience (Would they use it again? 
Would they recommend it to others?) and do 
they believe that in principle, upon this 
experience, CAL is a useful tool? 

Evaluation was conducted by diabetic 
patients, randomly sought from those 
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attending the Diabetes & Endocrinology Day 

Centre at St Thomas’ Hospital, South 

London, over the course of six weeks. 

Participants were invited to spend a minimum 

of 30 minutes browsing diabCAL, with the 

researcher available to answer questions and 
allay any anxiety as necessary. A standardised 
spoken preamble preceded participants’ 
browsing of the system. Having browsed the 

system, participants were asked to complete a 

written questionnaire, designed to address the 

aims of the evaluation by eliciting: 

* Patients’ views on the ease of use of 
diabCAL in general, the aesthetics and 
ease of use of the interface, the ease of use 
of the navigation, the value and 
accessibility of the content, and the use of 
various media. 

e Patients’ attitudes to using this system 
specifically to provide education about 
diabetes and self-care. 

e Patients’ attitudes to using computer- 
aided learning in general to provide 
education about diabetes and self-care. 


Data collected using the questionnaire were 
supplemented with observation data. The 
researcher took observation notes as users 
browsed the system, with the focus on where 
problems were encountered or errors 
occurred. 


Findings 

A total of 11 people participated in the 
evaluation with a mean age of 38 years (range: 
17-63). Observation data were collected for 
all participants. One further gentleman did 
commence participation, but was forced to 
withdraw because he simply could not 
manipulate the mouse. Of the eleven 
participants, eight were female and three 
male. Of the eleven completing participants 
two “never” used a computer and two did so 
only “rarely”. The remaining seven used a 
computer “often”. Six users felt “very 
comfortable” using a computer; however, 
three participants reported feeling generally 
uncomfortable on the PC. 


Usability 

In terms of how easy to use the system was 
found to be initially for new users, the 
majority reported it to be very or reasonably 
easy, with 27 per cent (n = 3) of users giving a 
score of 1 (very easy) and 36 per cent (n = 4) 
giving a score of 2. Three participants (27 per 
cent) gave a score of 3 — equating to “quite 
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difficult” — and 1 (9 per cent) a score of 4, 
indicating they found it “very difficult”. 
Where users had problems initially, 
comments related in some cases to the system 
itself and in others to users’ lack of familiarity 
with PCs. For example, the participant who 
gave a score of 4 added that “the problem is 
understanding how to use the computer. 
Re-learning how to use the mouse made it 
easier.” Another simply commented that “I’m 
not computer-literate.” (Indeed, later in the 
questionnaire, this participant indicated that 
they “never” use a computer, and that they 
were “not at all comfortable” with 
computers.) However, in other cases, specific 
problems caused by the interface were to 
blame, for example, “the screen shot of the 
module menu [in the ‘getting around’ pages] 
was confusing — not sure whether to click 
that”. This, of course, was exactly the kind of 
comment required, providing clear diagnosis 
of a specific impediment to usability. 
Despite reporting that problems were 
encountered initially, participants were 
generally happier (in terms of the ease of use 
of the system) over the whole duration of their 
session. Indeed, when asked how easy to use 
they found the system overall, four 
participants (36 per cent) found the system 
“very easy” to use (score of 1), six participants 
(54 per cent) “quite easy” (score of 2), and 
only one user found the system “quite 
difficult” (score of 3). No one this time gave a 
score of 4 (very difficult). Four participants 
gave the same score for initial and general 
ease of use; however, five awarded a higher 
score (where “highest” is 1) for their general 
experience — thus indicating that they found 
the system became easier to use as their 
session continued. Interestingly, the one user 
who originally awarded a score of 4 was the 
only participant to move by more than one 
place — a leap of two places from a 4 to a 2. 
Comparisons of scores awarded for initial and 
general ease of use can be found in Table V. 
In following up this question about general 
ease of use, when asked what made the system 
difficult to use in general, comments again 


Table V Scores awarded for initial and general ease of use 
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varied from specific problems or 
misunderstandings (“broken links”, 
“occasionally how to go back to previous 
screen wasn’t obvious, although it wasn’t 
difficult to get back there”, “unsure how to 
get back/go on when finished navigating 
through a subject”); to basic difficulties 
engendered by users’ lack of familiarity with 
the PC ("mouse”, “I did not know how to use 
the computer”). Despite problems 
encountered, both initially and in general, all 
the participants reported that this did not put 
them off using the system. 

For one participant, there was nothing that 
would make the system easier to use — it was 
fine as it stood. However, from a list of 
options given in the questionnaire, including: 
* training: how to use a computer; 

* training: how to use the system; 

* clearer help files; 

* someone sitting down and showing me 
the first time; and 

* repeated/frequent use, 


all other participants identified items that 
would help with usability. Participants could 
select as many options as they wished. As 
many as seven (64 per cent) users felt that 
training about how to use the system would 
help (despite only four — 36 per cent — 
reporting earlier that they found the system 
difficult to use initially), and six (55 per cent) 
felt that training about how to use a computer 
was important. Five (45 per cent) felt that 
“Someone sitting down and showing me the 
first time” would be useful, and five that 
repeated or frequent use would make the 
system easier to use. Only one participant felt 
that clearer help files (the “getting around” 
pages) would improve ease of use. General 
comments included “larger text/larger 
screens” and “a quick session as at the 
hospital today”. 


Navigation 

In terms of navigating the system using the 
module menu and sub-topic lists, all users 
reported that this made sense. As many as 
four participants (36 per cent) claimed to 





Participant number 1 2 3 4 5 6 7 8 9 10 11 
initial usability 2 3 3 3 1 2 2 1 4 1 
General usability 2 2 2 2 1 1 3 1 2 1 
Change 1/1 = T T T = t l = T = = 


Note: 1 = very easy; 4 = very difficult 
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have “never” found themselves lost or 
confused about their location after a few 
clicks through the menu or through links 
(score of 4), and two “rarely” (score of 3). 
The remaining five found themselves lost or 
confused only “sometimes” (score of 2). No 
one was lost or confused “often”. 

When asked what strategies were used for 
reorientation when users did find themselves 
lost or confused, four used the “back” button 
to return to their previous location, two 
returned using the sub-topic list, and three 
went back using the module menu. One user 
consulted the map for orientation, and two 
just continued forward through the links 
available. One user added her own comment 
— she checked the control panel to see where 
she was. This was great news, for orientation 
was a planned supplementary benefit of the 
control panel. 


System content 

Data collected about the content of the 
system was encouraging, with eight users (73 
per cent) reporting to have found the 
information on diabetes “very useful” (score 
of 1), and the remaining three (27 per cent) 
deeming the information “quite useful” (score 
of 2). Similar numbers were attained about 
how interesting the information was found to 
be — seven users (64 per cent) found the 
information “very interesting” and six (55 per 
cent) “quite useful”. 


Balance of media 

Regarding which of the various media they 
would like to see more, users indicated that 
they felt the media were already well 
balanced. The inclusion of more video, 
graphics, quizzes and FAQs would please 
most users, and almost half would like to see 


Table VI Matrix of scores awarded for balance of media 








User no. Text Video Pictures FAQs Quizzes None 
1 e 6 
2 e e 
3 Cs) 
4 e ° e C 
5 e e ® 
6 e e 
7 ° e e ® 
8 e e e e e 
9 © e ° e 
10 e e 
11 ° e e 9 ° 
Total 5 9 7 7 6 0 
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more text. Video was the most popular 
choice, with nine users (82 per cent) 
expressing a wish to see more clips in the 
system, and this was followed by pictures and 
FAQs (seven users each — 64 per cent). Six 
users (55 per cent) wished to see more 
quizzes, and five (45 per cent) more text. The 
distribution of users’ opinions is shown in 
Table VI. None of the users questioned felt 
they wanted to see less of any of these 
elements. 


Using CAL to learn about diabetes 

The majority of users indicated that they felt 
CAL to be a useful tool for patient education, 
with 82 per cent of users (n = 9) feeling it to 
be “very useful”. In addition, seven users (64 
per cent) said that, “yes, definitely” they 
would want to learn about diabetes using 
diabCAL (score of 1). Three users (27 per 
cent) gave a score of 2 (equating to “yes, 
probably”), one of whom added that this was 
“only because I do not like computers”. 
Finally, one user gave a score of 3, but added 
that this was because “basic knowledge [was] 
already acquired” (What is Diabetes? and 
Practical Skills are modules more appropriate 
to people newly or relatively recently 
diagnosed). 

If it were readily available, eight patients felt 
they would use the system often or very often. 
Only three felt they would seldom use the 
system — one who indicated above that she 
already had all the basic knowledge presented 
in these two modules, and two who had given 
high scores (1 and 2 respectively), indicating 
they would want to use the system to learn 
about diabetes. The findings in this section 
suggest that all the participants felt the system 
was of some value, even if not directly to 
themselves. Indeed, all participants agreed 
that they would recommend the system to 
someone else with diabetes. 


Observation findings 
On the basis of observation, specific points 
were noted at which users got stuck, or where 
the system misbehaved (due to technical 
bugs), and specific behaviours were noted, 
such as how users navigated backwards, and 
how they reacted to the control panel buttons. 
Despite any problems users encountered, 
these were not sufficient to deter any of them 
from using the system, and participants were 
generally enthusiastic. Indeed, one user, who 
had no previous computer experience, was 
especially enthusiastic and expressed that, if 
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the system were readily available, she would 
expect to get great use out of it and compared 
the system favourably with other patient 
information available. 


Discussion of findings 


Overall, users’ responses to the system were 
encouraging; although some participants did 
struggle initially, all were able to overcome 
problems, and several were enthusiastic about 
the experience, and about the prospect of 
such a system being available for patient use. 
The observed experiences of users and their 
subsequent responses suggest that the system 
is learnable. For example, although locating 
the module menu had, in the first instance, 
caused problems for some users, all reported 
that the method of navigation — using the 
module menu and the sub-topic list - made 
sense. Further, scores awarded for the general 
ease of use of the system were, for the most 
part, higher (better) than scores for initial ease 
of use. This finding was confirmed with the 
observation of users as they browsed the 
system: initial difficulties were overcome. 

In terms of the error rate, users were 
observed to make some errors (and some 
errors are to be expected), but recovery from 
error was found to be generally quite easy. 
The investigator — on hand and observing 
users as they browsed — was called upon only 
a few times to assist; in each case the problem 
was technical and bugs were therefore 
identified which required fixing. 

For users to be satisfied the system should 
be pleasant to use, so people should like it 
(Nielsen, 1993). Most users were, overall, 
satisfied with the system, and very 
encouraging responses were seen in terms of 
how participants felt about using diabCAL to 
learn about diabetes. One participant in 
particular, who reported that she was not 
computer-literate — and indeed had to be 
shown how to use the mouse — and did not 
feel at all comfortable with computers, was 
very enthusiastic. She reported that she would 
“definitely” use the system again if it were 
available, and would “definitely” recommend 
it to someone else with diabetes. She also 
commented that she “would be very 
interested to see it when it has been finished; 
there were other topics I would have liked to 
have read about”. Other informal comments 
to the researcher indicated similar levels of 
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satisfaction from several participants, and one 
user, a lecturer in computing, was very 
complimentary. 

In terms of the system tools, the Map, 
Resources, References and “getting around” 
pages were deemed invaluable by some. 
However, the glossary scored poorly for 
usefulness. This was contrary to expectations, 
since it was expected that the glossary would 
be highly valued. It is possible that users 
simply did not require access to the glossary 
in a short session such as this (30 minutes) 
and with the quite basic material that was 
covered here. Despite users feeling that the 
glossary is not one of diabCAL’s more useful 
tools, it remains available, if only for the sake 
of inclusivity. 

Modifications made on the basis of user 
feedback were generally trivial — the 
evaluation pointed out a few small errors in 
the text, and highlighted where there were 
invalid hyperlinks. Similarly, observation of 
users highlighted a technical flaw in the 
system map when viewed in a pop-up 
window, a problem that was easily rectified. 
Other than the modification of the navigation 
instructions, on the basis of user feedback, no 
further fundamental changes were necessary. 
However, that one participant chose to 
withdraw because he had such difficulty 
manipulating the mouse raised the option of 
touch-screen technology, in order to broaden 
the appeal of the system to older patients, as a 
serious item for consideration. 


Conclusions 


Evaluation is an integral part of the iterative 
development process employed in creating 
diabCAL, and the formative evaluations 
discussed here informed system development 
as it progressed. These formal evaluations 
were conducted variously by domain experts, 
who validated the structure and content of the 
system, and by representatives of the system’s 
two audiences, who provided vital feedback 
on usability. Verification of the concept maps, 
evaluation of navigation and content 
evaluation have all been important 
components of the evaluation process. Each 
has contributed to the shape of the finished 
prototype modules and has provided evidence 
in support of the overall system architecture. 
The usability assessment has contributed 
both by providing information as to how the 
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system might be improved at the level of 
formative evaluation, and by contributing at 
the level of summative evaluation in terms of 
user acceptance. With the two prototypical 
modules evaluated in this way, development 
of further modules can proceed and 
summative evaluation, looking at usefulness 
in terms of learning and motivation, can be 
conducted at a later stage. 
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Introduction 


Research evaluation is currently enjoying a 
favoured position in many countries as 
research budgets expand but there is also 
pressure from funders to ensure value for 
money (Balter, 2000; Hinde, 1997; 
Palmerini, 1999; Saegusa, 1999). In several 
European countries, for example, there is a 
move to more competitive funding both of 
laboratories and of individual research _ 
projects (Abbott, 1993; Allakhverdov, 1995; 
Goddard, 2000; Stone, 1996). The ultimate 
test of the utility of research is that it will give 
rise to new products and processes or, in the 
biomedical field, to better health for man and 
animals. However, the route to these tangible 
benefits is often lengthy, especially for basic 
research (Cerf, 1996; Martin and Salter, 
1996; Patel et al., 1999), so that proxy 
indicators are needed to show whether given 
expenditures have been worthwhile. The 
main proxy indicator is the esteem in which 
research is held by other researchers, which 
can be measured by the counting of citations. 
However, these suffer from some 
disadvantages: 

* they are somewhat costly to determine — 
typically about US$1 per paper; 

« it takes time for them to accumulate, so 
that fair comparisons can be made — 
perhaps five years; and 

* comparisons between different fields and 
sub-fields are difficult because citation 
norms differ greatly. 


An alternative proxy indicator of research 
quality is the prestige of the journal in which it 
has been published, which can conveniently’ 
be measured, at least in most areas of 
biomedicine, by citation counts. In effect, the 
judgements of editors and referees in selecting 
papers for publication can be used as a 
measure of quality (Lewison and van Rooyen, 
1999). Correspondence between journals’ 
mean citation scores and the actual numbers 
of citations received by papers (Lewison, 
1998} would indicate that the two measures 
are in reasonable agreement and that a 
ranking on the basis of the former criterion 
would be useful as a means to evaluate 
different groups of research papers. 

In some ways, the analysis of the factors 
that lead to high quality research is 
comparable with epidemiology. In this 
subject, the factors that lead to ill-health are 
analysed. The problem in both analyses is that 
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there may be missing factors, some of which 

are correlated with the ones used for analysis, 

and which can confound the apparent 
conclusions. For example, ill-health is 
positively associated with smoking, a poor 
diet, lack of exercise and a low-status job 

(Clarke et al., 1998; Brunner et al., 1999; 

Hemingway and Marmot, 1999; Marmot and 

Bobak, 2000). But each of these factors is 

strongly associated with the three others and 

indeed with yet more factors that can 
plausibly lead to ill-health, such as low 
income, low education level, poor housing 
and living in a polluted area. If public health 
measures are to be soundly based, it is 
essential that the individual effects of all these 
factors, and probably many more, are teased 
out, so that policies can be targeted on the 
most important ones. 

Similarly, the factors that lead to high 
quality research may plausibly include: 

e the subject area of the research; 

* whether it is applied or basic (in 
biomedicine, clinical or basic); 

* whether it is carried out in an industrial or 
academic setting (in biomedicine, in a 
hospital or a university); 

* whether the team of authors is large and 
multi-disciplinary or small; 

« whether there is inter-laboratory 
collaboration; 

* whether there is international 
collaboration and, if so, with which 
country or countries; 

* whether the research has been reviewed 
by a peer-review committee for funding 
and, if so, by how many; 

whether the research has been funded by 
a particular sector (government, private- 
non-profit, industry); and 

* the city in which the research has been 
conducted. 


It is the aim of this paper to show how such an 
analysis can be undertaken for one relatively 
large subject area and group of papers (cancer 
research in the UK from 1988 to 1998). 
Although numerical results will be given, they 
are intended more as illustrative examples to 
show the general principles of the analysis and 
to show trends than as definitive values. The 
results will show in particular the dangers of 
making judgements on inadequate data and of 
making inappropriate comparisons. These 
cannot be too highly stressed. There are 
increasing numbers of cases where 
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researchers have rejected the results of 
bibliometric or other quantitative studies, 
because they do not, in their view, reflect the 
reality of their situation (Collins, 1991; 
Williams, 1998). These rejections could 
prejudice the use of such quantitative studies 
where they can be useful as a supplement to 
the subjective views of peer-review 
committees (Lewison et al., 1999). 

The main analysis in the paper is of the 
potential impact category of the journals in 
which the papers are published, treated as a 
proxy indicator of research quality and an 
“output factor”, and how it is influenced by a 
large number of independent parameters or 
“input factors”. It is correlated with citation 
scores for groups of papers (Lewison, 1998), 
but there is inevitably much scatter and it is in 
effect almost an independent measure of 
impact. Subsequently the reasons for the 
changes in many of the parameters of UK 
oncology research with time (more authors, 
more collaboration, more funding etc.) are 
explored by a similar use of regression 
analysis. This reveals the main drivers behind 
the changes that have occurred and which 
ones have not been important. 


Methodology 


The study was based on a set of 27,189 
papers extracted from the Science Citation 
Index (SCI) and the Social Sciences: 
Citation Index (SSCI) © The Institute for 
Scientific Information, CD-ROM version, for 
the years 1988-1998. Articles, notes and 
reviews were downloaded with an address in 
the UK and in the field of cancer research. 
This was defined (Lewison, 1999) by means 
of lists of specialist journals and title 
keywords, some in combination. This 
retrieval strategy or “filter” was developed in 
association with representatives of the Cancer 
Research Campaign, a leading UK cancer 
charity. The bibliographic data from the SCI 
and SSCI were supplemented by funding data 
obtained by looking up the papers in libraries. 
The papers were in fact a sub-set of the 
Research Outputs Database (ROD), which 
has been developed by the Wellcome Trust in 
order to track UK biomedical research and 
identify papers associated with particular 
funders (Dawson et al., 1998). 

The funding data included the identity of 
the funder(s), their category (e.g. government 
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agency, charity, foundation, pharmaceutical 
company), type of funding (e.g. intramural, 
extramural, personal) and country of the 
funder. In particular, funders were grouped 
into three main sectors - UK government, 
UK private-non-profit, and industrial. The 
UK addresses were all given their correct 
postcodes, if they were missing or incorrect in 
the SCI and SSCI. For the purposes of the 
current analysis, interest was focused on the 
first one or two letters of the postcode 

(e.g. B = Birmingham, CB = Cambridge, 
EC = London East Central) which designate 
the city. The setting of the research was 
characterised as clinical or academic 
according to whether certain key word 
contractions (HOSP, INFIRM, NHS for 
clinical; COLL, SCH, UNIV for academic) 
were present in the address field. Finally, the 
papers with foreign addresses were identified 
and the presence of 11 leading industrial 
countries was noted for each. 

The papers were classified by research level 
using a system developed by CHI Research 
Inc. Narin et al., 1976). They assign a level 
(1 = clinical observation, 2 = clinical mix, 

3 = clinical investigation, 4 = basic research) 
to almost all journals on the basis of expert 
opinion and journal-to-journal citation 
patterns — clinical journals cite ones more 
basic but not vice versa. The distribution of 
papers by research level varies greatly 
according to the sub-field of biomedicine: 
papers are mostly basic in genetics and mostly 
clinical in anaesthesiology. In cancer research, 
there is a good balance with 14 per cent basic 
research (RL = 4), 30 per cent clinical 
investigation (RL = 3), 37 per cent clinical 
mix (RL = 2) and 18 per cent clinical 
observation (RL = 1). Only 1.3 per cent of 
papers are unclassified. 

The main output measure used was the 
potential impact category of the journals in 
which the papers were published, again with 
four levels. These were based on five-year 
mean citation scores (Co.4 vaiues) for papers 
published in given years and were taken from 
the Journal Expected Citation Rates files 
(©The Institute for Scientific Information). 
Papers were grouped into levels according to 
these values, as shown in Table I. The 
examples given are the most-frequently used 
journals in each category; some are specialist 
cancer journals and some are general journals. 

The appearance of Brit. f. Cancer in two 
categories is not a mistake: the journal 
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Table | Classification of journals by potential impact category (PIC) 
according to their five-year mean citation scores, Co.4 





PIC Co.4 values Examples 
1 Below 6 Brit. J. Urology, Brit. J. Radiology, Clin. 
Radiology 


2 Between6and11 Brit. J. Surgery, J. Clin. Pathoi., Eur. J. Cancer, 
Brit. J. Cancer 

3 Between 11 and 20 Brit. J. Cancer, Int. J. Cancer, Carcinogenesis 

4 Above 20 Cancer Res., Oncogene, Lancet, Blood 





changed category, as its citation score dipped 

below 11 in 1990 and then rose again to more 

than 15 in 1994. This illustrates the point that 
journals do vary in their citation impact with 
time, depending on the editor, the current 
importance of the subject and their publishing 
policy. 

The “output” or dependent variable in the 
regression analysis was PIC, which could take 
the value 1, 2, 3 or 4. The “input” or 
independent variables used for the analysis 
were as follows: 

+ A= number of authors, obtained by 
counting hyphens in the author field; 

* D = number of addresses, one more than 
the number of slashes in the address field; 

¢ DF = number of non-UK addresses, 
obtained by subtracting the numbers of 
addresses in England, Wales, Sotland and 
Northern Ireland from D; 

¢ FF, F2 = number of funding bodies and its 
square, obtained by counting the number 
of entries in the funding field; 

* YR = year of publication — 1987; 

* RL, RL2 and RL3 = research level, its 
square and its cube (these three 
coefficients allow for PIC, the dependent 
variable, having an arbitrary relationship 
with RL); 

* HOSP, ACAD = 1 or 0, depending on 
the presence of address key contractions; 

* MRC, CRC, ICRF = 1 or 0, depending 
on the presence of three leading funders 
of cancer research (the Medical Research 
Council, the Cancer Research Campaign, 
and the Imperial Cancer Research 
Fund[1]); 

* GOV, PNP, IND, NON = I or 0, 
depending on the presence of UK 
government, UK private-non-profit, 
industrial funding or no funding 
acknowledged; 

+ 20 postcode areas = 1 or 0, depending on 
whether they were represented in the 
postcode(s) for each paper; 
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* 11 country codes = 1 or 0, depending on 
whether they were present in the address 
field. 


The numbers of authors, addresses, foreign 
addresses and funding bodies were limited 
respectively to 9, 6, 3 and 5, because a few 
papers had very large values of these 
parameters and they would have distorted the 
analysis. Altogether, there were 49 
independent variables used in the analysis. 


Results 


The main results of the analysis are presented 
in Tables II and III. Table II shows the 
analysis by the principal non-geographic 
parameters — authors, addresses, funding and 
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year. The effect of research level is shown as 
the value of the unstandardised coefficient for 
each RL, including the constant term; 
although the coefficients in the cubic equation 
are all highly significant (< 0.1 per cent), it is 
not possible to show the statistical 
significance of each RL value. Overall, the 
contribution of the independent variables to 
the explanation of the variance of PIC was 31 
per cent. In the Tables, only the statistically 
significant coefficients are shown. 

The effect of research level on PIC should 
be seen more in terms of the overall 
distribution of papers than in terms of mean 
values, as PIC is a categorical variable, albeit 
one representing a continuum of values. 
Figure 1 shows that there are big differences 
between papers at the different research 
levels. 


Table Il SPSS analysis of effects of independent variables on PIC: authors (A), addresses (D and DF), funding (F) 


and individual funders 


Parameter N papers 
Authors, A 

Addresses, D 

Foreign addr, DF 6,258 
RL = 1 (clinical) 4,821 
RL = 2 10,076 
RL = 3 8,130 
RL = 4 (basic) 3,817 
Year, YR 

Setting, HOSP 19,064 
Setting, ACAD 16,755 
Funding, F 

Funding, GOV 5,429 
Funding, IND 3,311 
Funding, NON 9,364 
Funding, MRC 3,427 
Funding, CRC 5,363 
Funding, ICRF 2,835 


Significance (%) 


Coefficient Std error 

0.082 0.003 0.000 
-0.040 0.007 0.000 
—0.031 0.011 0.613 

0.793 

0.976 

1.270 

1.702 

0.022 0.002 0.000 
-0.043 0.013 0.131 

0.070 0.012 0.000 

0.093 0.020 0.000 
-0.145 0.023 0.000 
-0.113 0.018 0.000 
-0.173 0.026 0.000 

0.243 0.025 0.000 

0.104 0.018 0.000 

0.209 0.021 0.000 


Table lil SPSS analysis of effects of geographical presence in the address field of UK oncology papers, 1988-1998, 





on PIC 

Parameter N papers 
London W 2,818 
London WC 2,662 
London SW 1,915 
Oxford OX 1,726 
Cambridge CB 1,588 
London NW 835 
Liverpool L 805 
Bristol BS 732 
USA 2,178 





Coefficient Std error Significance (%) 
0.125 0.018 0.000 
0.150 0.019 0.000 
0.112 0.021 0.000 
0.131 0.022 0.000 
0.144 0.024 0.000 
0.156 0.031 0.000 

-0.122 0.031 0.009 
—0.090 0.033 0.579 
0.250 0.024 0.000 
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The effect of the number of authors per 
paper, shown as a coefficient of +0.082 in 
Table II, is equally dramatic, as Figure 2 
shows. It is the strong correlation (Figure 3) 
between numbers of addresses, D, and 
numbers of authors, A, and also with the 
numbers of funders, F, that led to the 
supposition that more addresses meant higher 
quality research in earlier work (Narin et al., 
1991; Glanzel, 2000). However, the negative 
coefficient for D in Table H shows clearly that 
this theory is erroneous. There is also a 
negative coefficient for DF, although from 
Table III it appears that collaboration with 
US researchers is very helpful in terms of 
publishing in high-impact journals. 

In view of the amount of labour involved in 
looking up every paper to determine its 
funding, it is reasonable to ask whether the 
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relationships of PIC with the other 
independent variables shown in Tables II and 
III actually depend on the availability of 
funding data. The SPSS analysis was, 
therefore, repeated with the nine funding 
variables removed (F and F2; GOV, PNP, 
IND and NON; MRC, CRC and ICRF). The 
results for the independent variables are 
shown in Tables IV and V. 

The absence of funding information makes 
little difference to the relationship between 
the estimated value of PIC and the non- 
geographic independent variables: there is 
naturally a larger coefficient for A but the 
coefficient of D is still negative and indeed 
almost unchanged. With regard to the 
geographical dependence (Table V compared 
with Table IID, there are now several other 
cities (Manchester, Sutton & Cheam, 


Figure 1 Distribution of papers by potential impact category (PIC; 1 = low, 4 = high) for UK oncology papers at 


four research levels (RL; 1 = clinical, 4 = basic), 1988-1998 
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Figure 2 Distribution of papers by potential impact category (PIC; 1 = low, 4 = high) for UK oncology papers with 


different numbers of authors, A, 1988-1998 











O%PIC1 O %PIC2 





m %PIC3 E %PIC4 

















The publication of cancer research papers in high impact journals Aslib Proceedings 
Grant Lewison Volume 55 « Number 5/6 - 2003 - 379-387 





Figure 3 Relationship between numbers of addresses, D, and mean numbers of authors per paper, A, and numbers 
of funders per paper, F, for UK oncology papers, 1988-1998 
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Table IV SPSS analysis of effects of independent variables on PIC: authors (A), addresses (D and DF), research level 
(RL), time and setting: without funding data 
Parameter N papers Coefficient Std error Significance (%) 
Authors, A 0.105 - 0.003 0.000 7 
Addresses, D -0.039 0.007 0.000 d 
Foreign addr, DF 6,258 -0.012 0.011 ns. 
RL = 1 (clinical) 4,821 1.075 
RL = 2 10,076 1.483 
RL = 3 8,130 1.832 
RL = 4 (basic) 3,817 2.177 
Year, YR 0.020 0.002 0.000 
Setting, HOSP 19,064 —0.098 0.013 0.000 
Setting, ACAD 16,755 0.087 0.012 0,000 
Table V SPSS analysis of effects of geographical presence in the address field of UK oncology papers, 1988-1998, f 
on PIC: without funding data \ 
a a A 4 
Parameter N papers Coefficient Std error Significance (%) 
London W 2,818 0.171 0.018 0.000 
London WC 2,662 0.255 0.019 0.000 
London SW 1,915 0.124 0.021 0.000 
Manchester M 1,906 0.049 0.022 2.448 
Sutton & Cheam SM 1,718 0.105 0.023 0.001 
Oxford OX 1,726 0.264 0.022 0.000 
Glasgow G 1,601 0.086 0.023 0.027 
Cambridge CB 1,588 0.262 0.023 0.000 
Edinburgh EH 1,415 0.098 0.025 0.007 
Birmingham B 1,315 0.060 0.026 1.826 i 
London NW 835 0.195 0.031 0.000 — 
Liverpool L 805 -0.129 0.032 0.005 
Bristol BS 732 -0.094 0.033 0.491 
USA 2,178 0.286 0.024 0.000 
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Glasgow, Edinburgh and Birmingham) that 
appear to have a positive influence on the PIC 
values of the oncology papers. This must be 
because they are successful in gaining external 
funding for their work and the positive effect 
on PIC that this brings is visible. However, 
the cities listed in Table III as having a 
positive coefficient do well, even when 
account is taken of their funding success. 
There are also two cities (Liverpool and 
Bristol) that appear to have a negative effect 
on PIC values by their presence in the address 
field, and this is so, whether or not account is 
taken of funding data. Again, only the USA 
appears to have a positive effect by its 
presence in the address field. 


Discussion 


The results presented above have provided 

several important findings: 

* The negative effect of numbers of 
addresses on the impact-category of 
journals in which research papers are 
published, which was found for 
gastroenterology, also applies to the larger 
field of cancer research. 

* International collaboration (except with 
the USA) also has a negative impact on 
journal impact category, mainly because 
the apparently positive effect is actually 
attributable to the additional funding, 
which more than counters the negative 
effect of collaboration across frontiers. 

e Clinicians tend to publish their papers in 
journals of much lower impact category 
than basic researchers in a field and this 
should be taken fully into account when 
comparisons between groups are being 
made. In particular, cancer research in 
hospitals is published in lower impact 
journals than ones from academia. 

* The number of authors on a paper plays a 
major role in determining the impact 
category of the journal in which it is likely 
to be published. 

e There is a small but positive drift in the 
impact category of journals in which 
cancer research is being published, 
probably because of the tendency for 
highly cited journals to increase in size 
and vice versa. 

* Jt is possible to compare the outputs of 
research from different cities (read, 
universities) on a fair basis and to identify 


Volume 55 - Number 5/6 - 2003 - 379-387 


those whose work is of a superior 
standard and those whose work appears 
to be of lower potential impact. However, 
such distinctions depend to some extent 
on a university’s success in gaining 
external funding, which could also be 
seen as an outcome measure. 


Although the data set has been used to 
evaluate PIC as an output measure, we can 
also use it to investigate the causes behind the 
steady change in the character of UK 
oncology research with time seen in Table VI. 

Evidently the research has become more 
multi-disciplinary, slightly more basic, with 
more papers specifically funded and much 
more foreign collaboration. Each of these 
variables can be considered as being the 
dependent variable, with all the others, as well 
as setting, year and number of addresses, 
being the independent variables. The results 
for authors, funding level, research level and 
foreign collaboration in turn are shown in 
Tables VII to X. 

The main drivers behind the increase in 
team size have been the need to get funding 
(more F) and more collaboration, particularly 
in the UK (more D, but DF not important). 
Research level plays little part but clinical 
team sizes seem to have increased more than 
basic ones. 

This shows that the primary drivers behind 
the apparent increase in funding are the larger 
research teams and increased foreign 
collaboration, and that funding has not 
inherently been more plentiful in recent years. 
Academics are more likely to be funded than 
clinicians in hospitals and there is a strong 
preference by funders for work at clinical 
investigation level, followed by basic research. 

The effect of collaboration on research level 
is divided: foreign collaboration tends to 
make it more basic but domestic collaboration 
is more clinical. Not surprisingly, hospital 
settings lead to more clinical work and 
academic ones to more basic work. But the 
biggest driver has been funding: the receipt of 
specific funding seems to make the work 
much more basic. 

The biggest influence on foreign 
collaboration has been the growing practice of 
domestic collaboration. The research level 
has had rather little influence, although basic 
research is more likely to lead to foreign 
collaboration, as is work in academia. 
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Table VI Variation with time of parameters associated with UK oncology research: authorship, A; funding, F; foreign 


collaboration, DF and research level, RL 








Period Papers Mean A 
1988-1990 6,549 4.06 
1991-1993 7,550 4.50 
1994-1996 8,116 5.07 
1997-1998 4,974 5.37 





Table VII SPSS analysis of effects of independent variables on team size 
{authors per paper, A) 





Parameter Coefficient Std error Significance (%) 
Year, YR 0.034 0.003 0.000 
Addresses, D 0.997 0.010 0.000 
Foreign addr, DF —0.001 0.016 ns. 
Funding, F 0.476 0.019 0.000 
Funding”, F2 ~0.046 0.004 0.000 
Setting, HOSP 0.244 0.025 0.000 
Setting, ACAD —0.064 0.022 0.358 
RL = 1 (clinical) 1.098 

RL = 2 1.320 

RL = 3 1.431 

RL = 4 (basic) 1.254 


Table VII SPSS analysis of effects of independent variables on funding 
level (funders per paper, F) 


Parameter Coefficient Std error Significance (%) 

Authors, A 0.148 0,004 0.000 

Addresses, D 0.078 0.009 0.000 
_ Foreign addr, DF 0.306 0.012 0.000 

Year, YR 0.003 0.002 ns. 

Setting, HOSP 0.423 0.018 0.000 

Setting, ACAD 0.127 0.017 0,000 

RL = 1 (clinical) 0.166 

RL = 2 0.663 

RL = 3 1.593 

RL = 4 (basic) 0.931 





Table IX SPSS analysis of effects of independent variables on research 
level (RL) 


Parameter Coefficient Std error Significance (%) 
Authors, A 0.021 0.003 0.000 
Addresses, D 0.078 0.006 0.000 
Foreign addr, DF 0.094 0.008 0.000 
Year, YR 0.010 0.002 0.000 
Funding, F 0.412 0.010 0.000 
Funding”, F2 ~0.050 0.002 0.000 
Setting, HOSP —0.498 0.012 0.000 
Setting, ACAD 0.107 0.011 0.000 





Mean F ‘Mean DF Mean RL 
1.17 0.36 2.28 % 
1.40 0.53 2,34 a 
1.54 0.73 2.44 
1.66 0.94 2.47 





Table X SPSS analysis of effects of independent variables on foreign 
collaboration (foreign addresses per paper, DF) 


Parameter Coefficient Std error Significance (%) 

Authors, A 0.000 0.002 n.s. ` 

Addresses, D 0.395 0.004 0.000 

Year, YR 0.005 0.001 0.035 

Funding, F -0.059 0.007 0.000 

Funding’, F2 0.030 0.001 0.000 

RL = 1 (clinical) -0.534 

RL=2 -0.503 

RL=3 —0.402 

RL = 4 (basic) -0.373 

Setting, HOSP -0.187 0.010 0.000 

Setting, ACAD 0.082 0.009 0.000 

It is worth noting that nearly all the i 
È 


coefficients in the Tables are highly 
statistically significant, because the data set is 
so large. It is likely that similar associations 
would be found for smaller data sets, e.g. in 
more specialist sub-fields or with papers from 
smaller countries. It is recommended that 
more analyses of research output should be 
conducted on a multi-factorial basis in order 
to elucidate the important underlying factors; 
this can probably be done satisfactorily 
provided that the data set contains at least 
several thousand papers. 


Note 


1 The Cancer Research Campaign and the Imperial 
Cancer Research Fund have now merged to form 
Cancer Research UK. 


References 


Abbott, A. (1993), “Czechs revise policy on research 
funding, aim for a more western approach. Prague 
institute leads the way", Nature, Vol. 361 No. 6408, 
pp. 104-5. 

Allakhverdov, A. (1995), “Russian science: president's 
council lambastes ministry”, Science, Vol. 268 No. 
5216, p. 1426. 

Balter, M. (2000), “New chief promises renewal and 
openness”, Science, Vol. 287 No. 5453, p. 567. 


386 


The publication of cancer research papers in high impact journals 


Aslib Proceedings 





Grant Lewison 


Brunner, E., Shipley, M.J., Blane, D., Smith, G.D. and 
Marmot, M.G. (1999), “When does cardiovascular 
risk start? Past and present socio-economic 
circumstances and risk-factors in adulthood”, 
Journal of Epidemiology and Community Health, 
Vol. 53, pp. 757-64. 

Cerf, V.G. (1996), “Research pays off", Science, Vol. 271 
No, 5243, 8 March, p. 1343. 

Clarke, R., Breeze, E., Sherliker, P., Shipley, M., 
Youngman, L., Fletcher, A., Fuhrer, R., Leion, D., 
Parish, S., Collins, R. and Marmot, M. (1998), 
“Design, objectives, and lessons from a pilot 25- 
year follow-up re-survey of survivors in the 
Whitehall study of London Civil Servants”, Journal 
of Epidemiology and Community Health, Vol. 52 No. 
6, pp. 364-9, 

Collins, P.M.D. (1991), Quantitative Assessment of 
Departmental Research: A Survey of Academics’ 
Views, SEPSU Policy Study No. 5, the Royal Society/ 
Fellowship of Engineering, London. 

Dawson, G., Lucocq, B., Cottrell, R. and Lewison, G. 
(1998), Mapping the Landscape: National 
Biomedical Research Outputs 1988-1995, Policy 
Report No. 9, The Wellcome Trust, London. 

Glanzel, W. (2000), “Science in Scandinavia: a bibliometric 
approach”, Scientometrics, Vol. 48, June, 
pp. 121-50. 

Goddard, A. (2000), “Research faces big overhaul”, THES, 
1 September, p. 4. 

Hemingway, H. and Marmot, M. (1999), “Evidence-based 
cardiology — psychosocial factors in the aetiology 
and prognosis of coronary heart disease ~ 
systematic review of prospective cohort studies”, 
British Medical Journal, Vol. 318, p. 1460. 

Hinde, J. (1997), “Institutes’ research pays”, THES, 4 July, 
p. 7. ; 

Lewison, G. (1998), “Gastroenterology research in the 
United Kingdom: funding sources and impact”, Gut, 
Vol. 43, August, pp. 288-93. 

Lewison, G. (1999), “The definition and calibration of 
biomedical subfields”, Scientometrics, Vol. 46 No. 3, 
pp. 529-37. 


Volume 55 : Number 5/6 - 2003 : 379-387 


Lewison, G. and van Rooyen, S. (1999), “Reviewers’ and 
editors’ perceptions of submitted manuscripts with 
different numbers of authors, addresses and 
funding sources”, Journal of Information Science, 
Vol. 25 No. 6, pp. 509-11. 

Lewison, G., Cottrell, R. and Dixon, D. (1999), 
“Bibliometric indicators to assist the peer review 
process in grant decisions”, Research Evaluation, 
Vol. 8 No. 1, pp. 47-52. 

Marmot, M. and Bobak, M. (2000), “International 
comparators and poverty and health in Europe”, 
British Medical Journal, Vol. 321, pp. 1124-8. 

Martin, B. and Salter, A. (1996), The Relationship between 
Publicly Funded Basic Research and Economic 
Performance, report for HM Treasury, Science Policy 
Research Unit, University of Sussex. 

Narin, F., Pinski, G. and Gee, H.H. (1976), “Structure of 
the biomedical literature", Journal of the 
American Society of Information Science, Vol. 27, 
pp. 25-45. 

Narin, F., Stevens, K. and Whitlow, E.S. (1991), “Scientific 
co-operation in Europe and the citation of 
multinationally authored papers”, Scientometrics, 
Vol. 21 No, 3, pp. 313-24. 

Palmerini, C. (1999), “Italy: university funding to be tied to 
performance”, Science, Vol. 284 No. 5415, 30 April, 
pp. 725-6. 

Patel, K., Thomson, A., Tysome, T., Utley, A. and Wojtas, 
O. (1999), “Nurturing the creative”, THES, 3 
December, p. 6. 

Saegusa, A, (1999), “Japanese labs balk at bid to boost 
external evaluation”, Nature, Vol. 397 No. 6718, 
4 February, p. 378. 


_ Stone, R. (1996), “Estonian researchers lead the way in 


science reform", Science, Vol. 273 No. 5284, 
4 October, pp. 29-30. 

Williams, G. (1998), “Misleading, unscientific and unjust: 
the United Kingdom's research assessment 
exercise”, British Medical Journal, Vol. 316, 
pp. 1079-82. 


387 


Literati Club 


Awards for Excellence 


Andrew Boyd 


City University, London, UK 
is the recipient of the Journal’s Outstanding Paper Award for Excellence for his paper 


“The goals, questions, indicators, measures (GQIM) approach to the 
measurement of customer satisfaction with e-commerce Web sites” 


which appeared in Aslib Proceedings: New Information Perspectives, Vol. 54 No. 3, 2002 


Andrew Boyd is currently pursuing a PhD at City University, where he is researching multi-source, 
multi-channel information-seeking behaviour within organisations, As a research student with the 
Centre for Information Behaviour and the Evaluation of Research (CIBER), he is a frequent 
contributor to Aslib Proceedings as well as to practitioner publications such as Content 
Management Focus, Knowledge Management and IT Consultant. 


He has ten years’ experience in strategic consultancy and technology management — specialising 
in the development of reporting and management information systems. He received his Master of 
Science in Information Systems from Drexel University and holds a Bachelor of Business 
Administration from Temple University. ; 


; 2 
| ; 388 





Emeral¢ 


a 


Fs 


ES 





Author and title index to volume 55, 2003 


Aslib Proceedings 








Author and title index 


Volume 55, 2003 








Abdullah, R., see Cox, A. 


3, p. 130 Accountability and demonstration of 
the value of information services in 
South African law firms 
Garratt, O. and du Toit, A. 


3, p. 181 After the disaster: drawing up the 
insurance claim 
Howes, R. 


5/6, p. 320 Availability and use of information 
touch-screen kiosks (to facilitate social 
inclusion) 

Boudioni, M. 





B 
Banyard, P., see Joinson, A.N. 


516, p. 367 Berridge, E.-J. and Roudsari, A.V. 
diabCAL: evaluating computer-aided 
learning for diabetes patient education 


1/2, p. 32 Beyond outputs: new measures of 
biomedical research impact 
Lewison, G. 


516, p. 320 Boudioni, M. 
Availability and use of information 
touch-screen kiosks (to facilitate social 
inclusion) 


5/6, p. 261 (The) British and their use of the Web 
for health information and advice: a 
survey 

Nicholas, D., Huntington, P., Gunter, 
B., Russell, C. and Withey, R. 

4, p. 203 (The) British Life and Internet 

Project: inaugural survey findings 
Gunter, B., Russell, C., Withey, R. 
and Nicholas, D. 





C 


4, p. 193. Can remote Internet voting increase 
turnout? 
Henry, S. 


516, p. 277 Characterising and profiling health 
Web user and site types: going beyond 
“hits” 
Huntington, P., Nicholas, D. and 
Williams, P. 





Volume 55 * Number 5/6 « 2003 » 389-392 


3, p. 117 Chua, A. 
Knowledge sharing: a game people 
play 

4, p. 234 Cloete, M. and Snyman, R. 
The enterprise portal — is it knowledge 


management? 


4, p. 223 Cloete, L.M., Snyman, R. and 
Cronjé, F.C. 
Training cataloguing students using a 
mix of media and technologies 


1/2, p. 52 Comparing the use of two DITV 
transmission services: same service, 
different outcomes 

Huntington, P., Nicholas, D. and 
Wiliams, P. 
3, p. 155 Cox, A. and Yeates, R. 
Library portal solutions 


4, p. 243 Cox, A., Patrick, K. and Abdullah, R. 
Seeding a community of interest: the 
experience of the knowledge library 
project 
Cronjé, J.C., see Cloete, L.M. 


D 


516, p. 357 Dandy, J. 
Public awareness of patient rights, 
consent forms and advance statements 


5/6, p. 367 diabCAL: evaluating computer-aided 
learning for diabetes patient education 
Berridge, E.-F. and Roudsari, A.V. 


1/2, p. 23 Digital information consumers, 
players and purchasers: information- 
seeking behaviour in the new digital 
interactive environment 

Nicholas, D., Dobrowolski, T., 
Withey, R., Russell, C., Huntington, P. 
and Wiliams, P. 


1/2, p. 43 Digital information provision via 
interactive television: understanding 
the digital consumer 

Gunter, B. 

516, p. 346 Digital interactive television: health 

information platform of the future? 
Gunter, B., Nicholas, D., 
Huntington, P. and Wiliams, P. 

1/2, p. 84 Digital journals, Big Deals and online 

searching behaviour: a pilot study 
Nicholas, D., Huntington, P. and 

Watkinson, A. 

Dobrowolski, T., see Nicholas, D. 


du Toit, A., see Garratt, O. 


Author and title index to volume 55, 2003 








E 


"E-mail overload in the UK workplace 


3, p. 166 
f Ingham, F. 


4, p. 234 (The) enterprise portal — is it 
knowledge management? 


Cloete, M. and Snyman, R. 





F 


3, p. 138 First steps towards providing the UK 
with health-care information and 
advice via their television sets: an 
evaluation of four Department of 
Health-sponsored pilot services 

Nicholas, D., Huntington, P., 
Williams, P. and Gunter, B. 





G 


Garratt, O. and du Toit, A. 
Accountability and demonstration of 


3, p. 130 


the value of information services in 
South African law firms 


Gunter, B. 

Digital information -provision via 
interactive television: understanding 
the digital consumer 

Gunter, B., see Nicholas, D. 

516, p. 346 Gunter, B., Nicholas, D., 
Huntington, P. and Williams, P. 
Digital interactive television: health 
information platform of the future? 
Gunter, B., Russell, C., Withey, R. 
and Nicholas, D. 

The British Life and Internet Project: 
inaugural survey findings 


1/2, p. 43 


4, p. 203 





H 
Harrison, J., see Hepworth, M. 


Si6, p. 304 Health information on the Internet: a 
qualitative study of NHS Direct 
Online users 
Williams, P., Huntington, P. and 
Nicholas, D. 


4, p. 193. Henry, S. 
Can remote Internet voting increase 


turnout? 


516, p. 290 Hepworth, M., Harrison, J. and 
James, N. 
Information needs of people with 
multiple sclerosis and the implications 
for information provision based on a 
national UK survey 


1/2, p. 64 Home electronic health information 
for the consumer: user evaluation of a 


DiTV video-on-demand service 


Aslib Proceedings 
Volume 55 - Number 5/6 - 2003 - 389-392 








Williams, P., Nickolas, D. and 
Huntington, P. 


3, p. 181 Howes, R. 
After the disaster: drawing up the 


insurance claim 


5/6, p. 277 Huntington, P., Nicholas, D. and 
Wiliams, P. 

Characterising and profiling health 
Web user and site types: going beyond 
“hits” 

Huntington, P., Nicholas, D. and 
Wiliams, P. 

Comparing the use of two DITV 
transmission services: same service, 
different outcomes 


1/2, p. 52 


Huntington, P., see Gunter, B. 
Huntington, P., see Nicholas, D. 
Huntington, P., see Williams, P. 





1/2, p. 13 Information metrics and user studies 
Tenopir, C. 


5/6, p. 290 Information needs of people with 
multiple sclerosis and the implications 
for information provision based on a 
national UK survey 
Hepworth, M., Harrison, J. and 
James, N. l 


3, p. 166 Ingham; J. 
E-mail overload in the UK workplace 








J 
James, N., see Hepworth, M. 


5l6, p. 313 Joinson, A.N. and Banyard, P. 
Seeking alcohol information on the 
Internet 

516, p. 334 Jones, R. 
Making health information accessible 
to patients 





K 


1/2, p. 5 Knowledge production, consumption 
and impact: policy indicators for a 
changing world 


Rowlands, I. 
Knowledge sharing: a game 


people play 
Chua, A. 


3, p. 117 





L 


1/2, p. 32 Lewtson, G. : 
Beyond outputs: new measures of 


biomedical research impact 


san 


Author and title index to volume 55, 2003 


Aslib Proceedings 





516, p. 379 Lewtson, G. 
The publication of cancer research 
papers in high impact journals 
3, p. 155 Library portal solutions 
Cox, A. and Yeates, R. 





M 


4, p.217 MacFarlane, A. 
On open source IR 


5/6, p. 339 McWhirter, G. 
The potential for using hospital 
bedside TV services in England to 
provide patients with access to health 
information 


5/6, p. 334 Making health information accessible 
to patients 
Fones, R. 
1/2, p. 18 (Mis)understanding the digital media 
revolution 
Withey, R. 


1/2, p. 75 Modelling the diffusion of specialised 
knowledge 
Warner, J. 


1/2, p. 23 Nicholas, D., Dobrowolski, T., 
Withey, R., Russell, C., Huntington, P. 
and Williams, P. 
Digital information consumers, 
players and purchasers: information- 
seeking behaviour in the new digital 
interactive environment 


1/2, p. 84 Nicholas, D., Huntington, P. and 
Watkinson, A. 
Digital journals, Big Deals and online 
searching behaviour: a pilot study 


516, p. 261 Nicholas, D., Huntington, P., Gunter, 
B., Russell, C. and Withey, R. 
The British and their use of the Web 
for health information and advice: a 
survey 

3, p. 138 Nicholas, D., Huntington, P., 
Williams, P. and Gunter, -B. 
First steps towards providing the UK 
with health-care information and 
advice via their television sets: an 
evaluation of four Department of 
Health-sponsored pilot services 
Nicholas, D., see Gunter, B. 
Nicholas, D., see Huntington, P. 
Nicholas, D., see Wiliams,`P. 


391 


Volume 55 - Number 5/6 > 2003 - 389-392 





o 


4, p. 217 On open source IR 
MacFarlane, A. 





P 
Patrick, K., see Cox, A. 


516, p. 339 (The) potential for using hospital 
bedside TV services in England to 
provide patients with access to health 
information 

McWhirter, G. 


516, p. 357 Public awareness of patient rights, 
consent forms and advance statements 
Dandy, J. 
5/6, p. 379 (The) publication of cancer research 
papers in high impact journals 
Lewison, G. 





R 
Roudsari, A.V., see Berridge, E.-}. 


1/2, p.5 Rowlands, I. 
Knowledge production, consumption 
and impact: policy indicators for a 
changing world 


Russell, C., see Gunter, B. 
Russell, C., see Nicholas, D. 


S 


4, p. 243 Seeding a community of interest: the 
experience of the knowledge library 
project 

Cox, A., Patrick, K. and Abdullah, R. 

5/6, p. 313 Seeking alcohol information on the 

Internet 
Joinson, A.N. and Banyard, P. 


Snyman, R., see Cloete, L.M. 


Snyman, R., see Cloete, M. 





T 


1/2, p. 13 Tenopir, C. 
Information metrics and user studies 


4, p. 223 Training cataloguing students using a 
mix of media and technologies 
Cloete, L.M., Snyman, R. and 
Cronjé, F.C. 





w 


1/2, p. 75 Warner, J. 
Modelling the diffusion of specialised 
knowledge 


Watkinson, A., see Nicholas, D. 
Williams, P., see Gunter, B. 


Author and title index to volurie 55, 2003 


Aslib Proceedings 





Williams, P., see Huntington, P. 
Williams, P., see Nicholas, D. 

5/6, p. 304. Williams; P., Huntington, P. and 
Nicholas, D. © 


Health information on the Internet: a 
qualitative study of NHS Direct 
Online users 7 


12, p. 64 Williams, P., Nicholas, D. and 
Huntington, P. 


Home electronic health information 


_ Volume 55 - Number 5/6 - 2003 - 389-392 


1/2, p. 18 


for the consumer: user evaluation of a 
DiTV video-on-demand service 
Withey, R. ; 
(Mis)understanding the digital media 
revolution 

Withey, R., see Gunter, B. 

Withey, R., see Nicholas, D. 





392 


Y 
Yeates, R., see Cox, A. 


ve 





Notes for contributors 





Aslib Proceedings: New Information 
Perspectives 


Copyright 
Articles submitted to the journal should be original 
contributions and should not be under consideration for any 


` other publication at the same time. Authors submitting articles 


for publication warrant that the work is not an infringement of 
any existing copyright and will indemnify the publisher against 
any breach of such warranty. For ease of dissemination and to 
ensure proper policing of use, papers and contributions become 
the legal copyright of the publisher unless otherwise agreed. 
Articles should be submitted by e-mail or on disc to: 


The Editor 

Professor David Nicholas, Head of Department, 
Department of information Science, City University, 
Northampton Square, London EC1V oHB, UK. 
E-mail: nicky@soi.city.ac.uk 


Editorial objectives 

Aslib Proceedings: New 
information Perspectives brings 
currency, authority and 
accessibility to the reporting of 
current research, issues and 
debates in the broad area of 
information work. Above all, the 
journal wishes to provide research 
and comment in a form that is 
easily and quickly understood: a 
fresh, rigorous, but unfussy, writing 
style is what is aimed for. The needs 
of busy practitioners (and 
academics) are very much in mind. 
Articles are refereed to meet the 
need for accredited and 
authoritative information. To 
ensure that this does not result in 
an unacceptable toss of currency, 
articles are speedily refereed by the journal's large, eminent 
and multi-disciplinary Editorial Board (assembled for just such 
a purpose). In some cases they may be referred to an external 
panel of consultants. The submission of research in progress 
articles is encouraged. 

The boundaries of information work are rapidly changing 
and widening, Aslib Proceedings has a role to play in the 
repositioning of the information profession by encouraging 
submissions from information managers, providers, packagers 
and consumers of all kinds — and especially those from the 
related fields of journalism, electronic publishing, 
communication and Internet studies. 


references 


Manuscript requirements 

As a guide, articles should be between 2,000 and 8,000 words 
in length. A title of not more than eight words should be 
provided. A brief autobiographical note should be supplied 
including full name, affiliation, e-mail address and full ` 
international contact details. Authors must supply an abstract 
of 160-150 words. Up to six keywords should be included which 
encapsulate the principal subjects covered by the article. 

Where there is a methodology, it should be clearly 
described under a separate heading. Headings must be short, 
clearly defined and not numbered. Notes or Endnotes should 
be used only if absolutely necessary and must be identified in 
the text by consecutive numbers, enclosed in square brackets 
and listed at the end of the article. 

All Figures (charts, diagrams and line drawings) and Plates 
(photographic images) should be submitted in both electronic 
form and hard copy originals. Figures should be of clear quality, 
black and white and numbered consecutively with arabic 
numerals. 

Electronic figures should be either copied and pasted or 
saved and imported from the origination software into a blank 


Final submission checklist 


Accurate, checked hard copy 
Abstracts and keywords 
Correct, complete Harvard 


Figures and illustrations in 


electronic and härd copy form ` 
Text on labelled disk or CD-ROM or 
as attached file if requested by 
editor ; i 
Completed/signed journal article 
record form - E 





Microsoft Word document. Figures created in MS Powerpoint 
are also acceptable. Acceptable standard image formats are: 
-eps, .pdf, .al and .wmf. If you are unable to supply graphics in 
these formats then please ensure they are..tif, jpeg, .bmp, 
-pcx, .pic, .gif or pct at a resolution of at least 3oodpi and at 
least 10cm wide. To prepare screenshots simultaneously press 
the “Alt” and “Print screen” keys on the keyboard, open a blank 
Microsoft Word document and simultaneously press “Ctrl” and 
“V” to paste the image. (Capture all the contents/windows on 
the computer screen to paste into MS Word, by simultaneously 
pressing “Ctrl” and “Print screen”.) 

For photographic images (plates) good quality original 
photographs should be submitted. If supplied electronically 
they should be saved as .tif or jpeg files at a resolution of at 
least 300dpi and at least 10cm wide. Digital camera settings 
should be set at the highest resolution/quality as possible. 

In the text of the paper the preferred position of all figures 
and plates should be indicated by typing on a separate line the 
words “Take in Figure (No.)” or “Take in Plate (No.)”. Supply 
succinct and clear captions for all figures and plates. 

Tables must be numbered consecutively with roman 
numerals and a brief title. In the text, the position of the table 
should be shown by typing on a separate line the words “Take 
in Table IV”. 

References to other publications must be in Harvard style 
and carefully checked for completeness, accuracy and 
consistency. This is very important in an electronic environment 
because it enables your readers to exploit the Reference 
Linking facility on the database and link back to the works you 
have cited through CrossRef. You 
should include all author names and 
initials and give any journal title in 
full. 

You should cite publications in 
the text: (Adams, 1997) using the 
first named author's name. At the, 
end of the paper a reference list in ` 
alphabetical order should be 
supplied: 

For books: surname, initials, 
(year), title of book, publisher, place 
of publication, e.g. Fallbright, A. and 
Khan, G. (2001), Competing 
Strategies, Outhouse Press, 
Rochester. 

For book chapters: surname, 
initials, (year), “chapter title”, 
editor's surname, initials, title of 
book, publisher, place of 
publication, pages, e.g. Bessley, M. and Wilson, P. (1999), 
“Marketing for the production manager”, in Levicki, J. (Ed.), 
Taking the Blinkers off Managers, Broom Reim, London, pp.29-33. 

For journais: surname, initials, (year), “title of article”, 
journal name, volume, number, pages, e.g. Greenwald, E. 
(2000), “Empowered to serve”, Management Decision, Vol. 33 
No. 5, pp. 6-10. 

For electronic sources: if available online the full URL should 
be supplied at the end of the reference. 


Final submission of the article 

Once accepted for publication, the final version of the 
manuscript must be provided, accompanied by a 3.5" disk, Zip 
disk or CD-ROM of the same version labelled with: disk format 
(Macintosh or PC); author name(s); title of article; journal title; 
file name. y ` 

Alternatively, the editor may request the final version as an 
attached file to an e-mail. 

Each article must be accompanied by a completed and 
signed journal Article Record Form available from the Editor or 
on http://www.emeraldinsight.com/literaticlub/jarforms.htm 

The manuscript will be considered to be the definitive 
version of the article. The author must ensure that itis 
complete, grammatically correct and without spelling or 
typographical errors. 

In preparing the disk, please use one of the following 
preferred formats: Word, Word Perfect, Rich text format or 
TeX/LaTeX. 

Technical assistance is available from Emerald’s Literati 
Club on http://www.emeraldinsight.com/literaticlub or by 
contacting Mike Massey at Emerald, e-mail: mmasey@ 
emeraldinsight.com 


Aslib Proceedings 


New Information 
Perspectives 


Volume 55 Number 5/6 2003 $ 
In 2003 this journal will consist of 6 printed issues delivered in 5 despatches 2 i 3 
+ Internet Online Publishing with Archive, Active Reference Linking, Emerald WIRE, Key Readings, i 
Research Register, Non-article Content, Institution-wide Licence, E-mail Alerting Service and Usage Statistics 





Contents 


Number 5/6 | 


Health informatics: inter-disciplinary and multi-disciplinary perspectives . 
Guest Editors: Peter Williams and Paul Huntington 


254 Access this journal online 
255 Abstracts & keywords 
258 Guest editorial 


261 The British and their use of the Web for health information and advice: a survey . 
David Nicholas, Paul Huntington, Barrie Gunter, Chris Russell arrd Richard Withey ae 


277 Characterising and profiling health Web user and site types: going beyond “hits” 


Paul Huntington, David Nicholas and Peter Williams : 


290 Information needs of people with multiple sclerosis and the implications for information soi oe 


provision based on a national UK survey t . ; iy ser 
Mark Hepworth, Janet Harrison and Nicole James : i : : à’ 





™ 


304 Health information on the Internet: a qualitative study of NHS Direct Online: users 
Peter Williams, Paul Huntington and David Nicholas 





313 Seeking alcohol information on the Internet . E 
Adam N. Joinson and Phil Banyard ; i wo 


320 Availability and use of information touch-screen kiosks (to facilitate social inclusion) 
Markella Boùdioni=- -~ 


334 Making health information accessible to patients 
Ray Jones Sa 





339 The: ‘potential for using hospital bedside v services in England to. provide patients with 


access to health information í af eS 
Gavin McWhirter oe Sena | 


346 Digital interactive television: health information platform of the future? 
Barrie Gunter, David Nicholas, Paul Huntington and Peter Williams 


357 Public awareness of patient rights, consent forms and advance statements 
Julia Dandy 


367 diabCAL: evaluating computer-aided learning for diabetes patient education 
Emma-Jane Berridge and AV, Roudsari 


379 The publication of cancer research papers in high impact journals 
Grant Lewison 


388 Awards for Excellence 
` 389 Author and title index to volume 55, 2003 





er S l www.emerald nsight. 


